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Chapter 1 

Introduction 



1.1 The coupon collector's problem 

The coupon collector's problem is one of the classical problems of probability theory. The simplest and 
probably original version of the problem is the following: Suppose that there are n coupons, from which 
coupons are being collected with replacement. What is the probability that more than t sample trials are 
needed to collect all n coupons? One of the first discussions of the problem is due to Polya [25]. It is 
brought up 7 times in Feller [13]. The problem has numerous variants and generalizations. It is related to 
urn problems and the study of waiting times of various random phenomena (e.g. [17], [16], [1]), etc. 

We shall be interested in the following version of the problem. A coupon collector samples with 
replacement a set of n > 2 distinct coupons so that at each time any one of the n coupons is drawn with 
the same probability 1/n. For a fixed integer m € {0, 1, . . . ,n — 1}, this is repeated until n — m distinct 
coupons are collected for the first time. Let W n , m denote the number of necessary repetitions to achieve 
this. Thus the random variable W n , m , called the coupon collector's waiting time, can take on the values 
n — m, n — m + l,n — m + 2, . . ., and gives the number of draws necessary to have a collection, for the first 
time, with only m coupons missing. In particular, W n _o is the waiting time to acquire, for the first time, a 
complete collection. 

The starting point in the study of the behavior of the distribution of the coupon collector's waiting 
time is the well-known equality in distribution ([13], p. 225) 

W n:m = X n / n + X(n_iy„ + ■ ■ • + X( m+ iy n , (1-1) 

where X n / n , Xr n _iy n , . . . , X( TO _|_i) /„ are independent random variables with geometric distributions per- 
taining to the success probabilities n/n,(n — l)/n, (m + l)/n, respectively, so that P{X/./„ = j} = 

(! " I) 5 " 1 !' 3 S N := {1, 2, . . .}, for every k € {m + 1, . . . , n}. 

Since the mean and variance of a geometric random variable with parameter p are 1/p and (1 — p)/p 2 
respectively, the mean and variance of the waiting time are 

" 1 

= fJtn(m) := E(W n , OT ) =n ^ -, (1.2) 

k—m-\-l 

and 

n , n-1 , 

al = al{m):=V^{W n>m )=n £ ^ = „ £ (1.3) 

fc— m+1 k—rn-\-l 

1.2 Limit theorems in the coupon collector's problem 

Different limit theorems have been proved for the asymptotic distribution of W n ^ n , depending on how m 
behaves as n — > oo. From now on all asymptotic relations throughout are meant as n — > oo unless otherwise 
specified. 
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The first result was proved by Erdos and Renyi [12] for complete collections when m = for all n € N, 
obtaining a limiting Gumbel extreme value distribution: 



n 

where the probability measure Gumbel(O) is defined to be the Gumbel distribution shifted by Euler's con- 
stant: 

Gumbel(0){(-oo, x}} = e - e ~ (x+7) , 

where 7 = hm„^oo (J2k=l \ ~ lo S n ) = 577215 . . .. 

This result was extended by Baum and Billingsley [8], who examined all relevant sequences of m. 
They determined four different limiting distributions: 

1. Degenerate distribution at 

Ti — Tn x5 
If — -= > 0, then W n m — (n — m) — > 0, 

that is the limiting probability measure is concentrated on 0. 

2. Poisson distribution 



If n ™ -> V2X, then W„ m - (n - m) Po(A), 

In 



where Po(A) is the Poisson distribution with parameter A defined by Po(A){fc} = 4fe A , k = 0,1,2,.. .. 

3. Normal distribution 

If — — > 00 and m — > 00, then ■ ^-N(0, 1), 

where N(0, 1) denotes the standard normal distribution, whose probability density function with re- 
spect to the Lebesgue measure is —}==e~ x / 2 , x £ R. 

4. Gumbel-like distribution 

If m = to, then — — — Gumbel(m), 
n 

where we call Gumbel(TO) the Gumbel-like distribution with parameter m, and define it to be the 
probability measure with probability density function with respect to the Lebesgue measure 

l e - (m+ i)(^-jj =1 1) e _ e -(-^-^ *) x e M. 
to! 



1.3 Aims of the thesis 



One of the aims of this thesis is to refine the limit theorems of the previous section. Our basic goal is 
to approximate the distribution of the coupon collector's appropriately centered and normalized waiting 
time with well-known measures with high accuracy, and in many cases prove asymptotic expansions for the 
related probability distribution functions and mass functions. The approximating measures shall be chosen 
from five different measure families. Three of them - the Poisson distributions, the normal distributions and 
the Gumbel-like distributions - shall be probability measure families whose members occur as limiting laws 
in the limit theorems of Baum and Billingsley. 

The fourth set of measures considered shall be a certain {71^. a : /i > 0, a > 0} family of compound 
Poisson measures which we now define. For each u > and a > let 7T M>a denote the probability distribution 
of Z\ + 2^2, where Z\ and Z2 are independent random variables defined on a common probability space, 
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Zi ~ Po(/x) and Z 2 ~ Po(a/2). Since Po(A), A > 0, has probability generating function exp{A(z — 1)}, the 
probability generating function of Z\ + 2Z 2 is 

g(z) := E (z Zl+2Z2 ) = E (z Zl ) E {{z 2 ) Z2 ) = e^" 1 ^^ 2 " 1 ) 

By the basic properties of probability generating functions, we see that Z\ + 2Z 2 does have a compound 
Poisson distribution, that is, it equals in distribution a random variable of the form X^fcLi^fci where 
N, Xi,X 2 , ■ ■ ■ are independent random variables given on a common probability space such that N has 
Poisson distribution and X 1; X 2 ,... are identically distributed, namely N ~ Po (p + |) and each Xj., 
fc = 1, 2, . . ., takes on the values 1 and 2 in the proportion /1 : § . 

The fifth set of approximating measures we consider shall be the family of Poisson-Charlier signed 
measures. For any positive real numbers A, . . . and SgN, the Poisson-Charlier signed measure 
v = v{\, a^', . . . ,a^) is a signed measure concentrated on the nonnegative integers defined by 

H.]} = Po{.?}(A) r£(-iya^C r (j, X)^j , j g N, (1.4) 

where 

fto,A)-.=E(l)( J >«-» M 

fc=0 v 7 v 7 

is the r-th Charlier polynomial ([9] p. 170). 

In the next chapter we explain the basic underlying ideas of the methods used in the proofs of the 
thesis, and list some of the important results we shall use. Then, each of the following five chapters is 
dedicated to the approximation of the coupon collector's waiting time with members of one of the five 
chosen measure families. 

The results of Chapter 3 were published in [30], those of Chapter 4 were published in [29]. The results 
of the first three sections of Chapter 5 can be found in [28], some details are contained in [27]. The results 
of Chapter 6 were published in [26]. 
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Chapter 2 



Methods to measure the closeness of 
probability distributions 



2.1 Probability metrics 

There are several ways of defining the distance of two probability distributions. (See e.g. [14] and [31].) 
Throughout this section let fj, and v be two probability measures on the measurable space (R, B) , where B 
denotes the <r-algebra of the Borel sets of the real line. Let X be a real random variable with distribution 
/z and distribution function F, and let Y be a real random variable with distribution v and distribution 
function G. For an arbitrary family % of bounded real measurable functions on the real line we define 



which we call the probability metric associated with the family of test functions "H, if H. is coarse enough to 
assure d^(/z, v) — => [i = v. 

In this thesis we shall be interested in the probability metrics resulting from % = {indicator functions of (-co, x], x G 
R} and H. = {indicator functions of all Borel sets}, which are known as Kolmogorov distance and total vari- 
ation distance respectively. 

Kolmogorov distance 

The Kolmogorov distance between \i and v is defined to be 



Clearly, this is exactly the supremum distance of the corresponding distribution functions: d^fiji/) = 
sup xeM |-F(o:) -G(x)\. 

Obviously, < <1k(h,v) < 1, <1k(h,v) = iff /i = v, and d-^^i.v) = 1 iff sup{x € R : F(x) < 1} < 
inf{x G R : G(x) > 0} or sup{x 6 R : G(x) < 1} < mf{x G R : F(x) > 0}. Since d K (; •) as a function of 
two variables is also symmetric and satisfies the triangle inequality, c?k(', •) is indeed a metric on the space 
of probability distributions on (R, B). 

Convergence in Kolmogorov metric is stronger than convergence in distribution, that is if /j, n , n G N, 
is a sequence of probability measures on (R, B) with corresponding distribution functions F n , n G N, such 
that d^Hnj/J,) — > 0, then \x n converges weakly to n, meaning that F n (x) — > F(x) for each x G R continuity 
point of F. The converse is not true in general. (See [14] p. 14 Theorem 6) One possible metric that metrizes 
weak convergence of probability measures on (R, B) is the Levy metric defined by 




(2.1) 



dx(A*, v) = sup kt((— co, x]) — v((— oo, x\) \. 



(2.2) 



inf{e > : F(x - e) - e < G{x) < F{x + e) + e, Vx G R}. 
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We have 

^lO, v) < d K (fi, v)< ( 1 + sup \F'(x)\ ) d L (p,, v), 

\ x£R J 

where the first inequality is true for any choices of fi and v (see [18] p. 34), while the second one holds 
true only if fi is absolutely continuous with respect to the Lebesgue measure (see [23] p. 43). This implies 
that the weak convergence fi n =t fx is equivalent to dj^(fx n , fi) — > in the case when the limiting probability 
distribution fx is absolutely continuous and has a bounded density function. 

Total variation distance 

The total variation distance between fx and v is defined to be 

chv{n,u) = sup \(jl(B) -v{B)\. (2.3) 

The definition above may be given in other equivalent forms. By [5] p. 253, if fx and v are both absolutely 
continuous with respect to a cr-finite measure A (for example A = fi + v) , and / and g are the densities of fx 
and v with respect to A, then 

d TV (n, v) = \fx{B ) - v(B )\, where B = {x G R : f(x) > g{x)} 

i r°° 

/OO 
min{/,.9}dA. 
-oo 

Later we shall be interested in the case when fx and v are the distributions of certain integer valued 
random variables X and Y defined on a common probability space (f2, A, P). If we choose A to be the 
cr-finite measure that puts unit mass on each of the integers, the formulas above yield 

d T v(/i, v) = \V{X G B ) - P{Y G S ) |, where B = {k e Z : P(X = k) > P(Y = k)} 

= 1 - ^min{P(X = k),P{Y = k)}. 

feez 

As in the case of the Kolmogorov metric, it is easy to see that < drrviHt ^) < 1, dTv(M, v) = iff 
/.t = u, and dxv(/J>, v) = 1 iff fj, and v are mutually singular. Since c/tv( - , •) as a function of two variables is 
obviously symmetric and satisfies the triangle inequality, g?tv('i ') is indeed another metric on the space of 
probability distributions on (R, B) . 

We note that d^{fi, v) < c^tvCa*) i 7 )- It follows by our remarks concerning convergence in Kolmogorov 
distance that convergence in total variation distance is stronger than convergence in distribution, that is if 
/i„, n € N, is a sequence of probability measures on (R, B) such that rfTv(/ i n J A*) — > 0, then \i n converges 
weakly to /i. It is easy to give an example showing that the converse of this statement is not true in general. 
We may take any sequence of discrete real valued random variables for which the central limit theorem 
holds true. In this case although the induced probability measures converge weakly to the standard normal 
distribution, the corresponding total variation distances all equal 1, simply because an absolutely continuous 
and a discrete probability measure are always mutually singular. However, if the probability distributions 
fi n , n G N, and fi are concentrated on a countable subset of R, then fi n fi implies dTv(/^n, /•*) — ► (see 
[14] p. 14 Theorem 6). 



2.2 The method of characteristic functions 

Let X be a real random variable defined on a probability space (0, A., P) with distribution fi = P o X^ 1 and 
distribution function F(x) = fJ>((— oo, x]), x € R. The characteristic function of X is the complex valued 
function 

/OO /"OO /-OO 

e itx fi(dx)= cos(tx)fi(dx) + i / sm(tx)fj,(dx), 
-OO J —OO J —oo 
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well-defined for all ( e 1. The characteristic function of any real random variable completely defines its 
probability distribution, that is there is a one-to-one correspondence between probability measures on (R, B) 
and characteristic functions ([13], Volume II. p. 508). 

One of the most important applications of characteristic functions is the study of convergence in 
distribution. The Continuity Theorem ([13], Volume II. p. 508) states that in order that a sequence {p, n }n£W 
of probability distributions converges weakly to a probability distribution p it is necessary and sufficient 
that the sequence {<p n }neN of their characteristic functions converges pointwise to a limit ip, and that if is 
continuous at the origin. In this case ip is the characteristic function of p. It follows that if X n is a real 
random variable with distribution p n , distribution function F n and characteristic function ip n , neN, and 
the same goes for X with p, F and ip, then 

X n ^ X p n =t (j, F n (x) — >• F(x),x G Cp <p n (t) -> <p(t),t G K, 

where Cf denotes the set of continuity points of F. 

From this it is clear that characteristic functions are an important tool for proving limit theorems. 
There are also classical results on characteristic functions which provide methods to refine limit theorems. 
One of these is Esseen's smoothing inequality ([24] p. 109), which applied on two probability distribu- 
tion functions, gives an upper bound on their Kolmogorov distance with the help of the difference of the 
corresponding characteristic functions. 

Esseen's smoothing inequality. If F is a nondecreasing function, G is a differentiable function of bounded 
variation and bounded derivative g, liniz-^-oo F(x) = lim^^-oo G(x) and lmx^oo F(x) = lim^^oo G(x), and 
if and ip are the Fourier-Stieltjes transforms of F and G respectively, that is 

/oo />oo 

e itx dF(x) and ip{t) = I c itx dG{x), t G R, 
-oo J —oo 

then for any T > we have 

bu P |F(x)-G(x)|<^- f 

where b > 1 is arbitrary and q, > is a constant depending only on b. 

An analogous result for comparing discrete distributions with the help of their characteristic functions 
is given in a recent paper of Barbour, Kowalski and Nikeghbali. We gather the results of Proposition 2.2., 
Corollary 2.3. and the formulas (3.15)-(3.17) on p. 11 in [6] in the following theorem. 



dt 



Cb- 



SUPx gR Iff 0*0 1 

T 



Theorem 2.2.1 (Barbour, Kowalski and Nikeghbali) Let /j, and v be finite signed measures on Z, with 
Fourier-Stieltjes transforms <f> and ip respectively, that is 

(f>(t) = ^e itfc /i{fe} and ip(t) = ^e ltk v{k], t G M. 

fcez feez 

Suppose that <p = (f>x and "0 = tpx f or some functions <p, if), x '■ K — > C, and that for some constants 
t a , 70, 7, p, rj > and j r , 9 r > 0, r = 1, 2, . . . , S, S G N, 



m)-if(t)\ <^7r|*| er +7o and \ x (t)\ < ^ pt , < |t| < t (2.4) 

m)-^(t)\ <n, t Q <\t\<TT. (2.5) 

Then 

s 

sup \p{k} - v{k}\ < a e T llr{p V 1) r ~ + «i77o + a 2 V, (2.6) 
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furthermore, if fx is a probability measure, then also 



d K (n, v) < inf 2\v\{(-<x>, a) U {b, oo)} + 2e ab (2.7) 

a<b 



and 



a<b 



2 d TV (fi, u) < inf \(b - a + 1) sup Hfc} - j/{fc}| + 6\u\{(-oo, a) U (6, oo)} + 4e ab , (2 



where ag r are positive constants depending on 8 r , r — 1,2, . . . , R, ol\ = ^ A a2 = ^ — w arl< ^ 

s 

£a6 := > y "e r 77r(p V 1) 2 + (6 - a + l)(ai77o + a 2 ??)- (2.9) 

r=l 

2.3 Stein's method 

Stein's method is a way of deriving explicit estimates for the closeness of two probability distribution. It 
was introduced by Charles Stein for normal approximation in [32] in 1972. We shall now review the basic 
idea of the method (see [3]). Let (j,q be a fixed probability measure on (R,£>), which we shall approximate 
with another probability measure /i on (R, B) . The error of the approximation will be measured in the 
probability metric du(-, ■) defined in (2.1), where H is a well-chosen fixed family of test functions. Stein's 
method consists of the following three steps: 

1. The Stein characterization of fiQ 

One needs to find a set of functions Jo C J := {/ : 1 -> 1 measurable} and a mapping T : — > J~ 
called the Stein operator for fiQ such that 

/oo 
Tf dfi = for all / G 7b, (2.10) 
-oo 

or equivalently for any real random variable X 

X - if and only if E(Tf(X)) = for all / G F Q . 

2. Solving the Stein equation 

For each test function h G % one needs to find a solution / = /(,£ F$ of the 

h(x)- hdfx = Tf(x), xeR, (2.11) 



Stein equation. If such an fh exists for each test function h G W, then writing the solution in the above 
Stein equation, integrating both sides of the equation with respect to /i and taking the supremum of 
the absolute values of both sides over all test functions yields 



dub*, Ho) = sup 
hen 



h dfj, — / h d/io 



sup 

hen 



Tfhd/i 



that is for any random variable X ~ fi we obtain 



Mo) = sup |E(T/ h (X))| . (2.12) 
hen 



3. Bounding sup heH \E(Tf h (X))\ 



We have obtained a formula in (2.12) that expresses d-u{fi, /zo) as a supremum of certain expectations. 
In the formula the distribution Mo is only present implicitly through the choices of the functions Tfh- 
To bound dn(fi,, no) we need to give estimates for the expectations E(Tfh(X)), which are surprisingly 
easier to bound than the original defining formula of the distance, if the Stein operator T was chosen 
in a clever way. We note that being able to give good approximations to the expectations E(T fh(X)) 
depends heavily on the properties of the solutions fh ■ 
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We conclude that the key point in the procedure proposed above is to chose a good Stein operator for 
n °t only does T need to characterize /iq as given in (2.10), it also has to yield a Stein equation (2.11) 
that has a solution fh for each test function h € H, moreover these solutions need to have nice properties. 

In this thesis we shall apply results proved by Stein's method to approximate the appropriate function 
of the coupon collector's waiting time with a compound Poisson random variable, and we shall apply the 
method directly to obtain a Poisson approximation error estimate for the same waiting time. For later use, 
we now collect the basic results on Poisson approximation in total variation distance with Stein's method. 

Poisson approximation with Stein's method 

Stein's method was first extended to Poisson approximation by Chen in [10]. The theory was further 
developed by Barbour, Hoist, Janson and others (see [3] and [5]). 
The Stein operator for Po(A) is 

T : Jo := {/ : ^+ -> M bounded} -> T := {/ : Z+ -> K}, (T/)(fc) = A/(fc + 1) - kf(k). 

It can be proved ([3] p. 65) that this operator T characterizes Po(A) in the required way, that is for any 
probability measure fi on Z + 

H = Po(A) / Tfdn = for all / e F . 

It can also be proved ([3] p. 66) that for each Ka indicator function of A C Z+, the Stein equation 

Xf(k + 1) - kf(k) = h(k) - f T/dPo(A) (2.13) 

has a solution fh = Ja, an d 

sup \h{k)\<mm\l,Jl-\ (2.14) 



A-e: 



eA 



and 



The method yields the formula 



sup \f h (k + l)-f h {k)\ < 



-A 



<*rv(Z>P0,Po(A))= sup \E{Xf A {X)-Xf A {X)}\. (2.15) 

2.4 Couplings 

Let X and Y be random variables defined on the probability spaces (f2i,>ti,Pi) and (^2, ^2,^2) respec- 
tively. A coupling of X and Y is a pair of random variables X' and Y' that are defined on the same 
probability space (CI, A, P), and such that X=X' and Y=Y'. Clearly the coupling of X and Y only depends 
on the distribution of these random variables. It will be useful for us to think of a coupling the following way: 
given two probability distributions /ii and ^2 on (R, B) , a coupling of these probability measures means the 
construction of a probability space (fl, A, P) and a random vector (X 1 , Y') on this probability space whose 
[i distribution on (M 2 ,S 2 ) has marginals /ii and fi2- 

Couplings are used in a vast variety of proofs (see e.g. [20] and [33]). In each of them the basic 
underlying idea is to construct a suitable coupling (X',Y') such that X' and Y' have the dependence 
structure most adequate for handling the problem considered. In this thesis we shall use the coupling 
method to give estimates for the total variation distance of certain distributions. We now present the basic 
relation between couplings and total variation distance. 

The coupling inequality 
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If (X' ,Y') defined on (CI, A, P) is a coupling of the random variables X and Y, then 

d T y(X,Y) <P(X'^Y'). (2.16) 

Since the proof of the coupling inequality is quite simple and short, we include it here: 

d TV (X, Y) = sup \P(X' G B) - P(Y' G B)\ 
BeB 

< sup \P(X' G B,X' = Y') - P(Y' G B, X' = Y')\+ 
BeB 

+ sup \P(X' €B,X'jL y') - P(Y' eB.lV Y')\ 
BeB 

<P(X' ^Y'). 

It can be proved that there always exists a coupling for which there is equality in (2.16) ([20] p. 19). 

Couplings to bound d-rv(W, W + 1), where W is a sum of independent integer valued random 
variables 

Let X\, Xi, ... be independent integer valued random variables, and let W n = $3j=i -^-h n = 1,2, . . .. 
We are interested in estimating the total variation distance d,Tv(W n ,W n + 1), and there is a technique 
involving couplings for this purpose ([20] Chapter 3). 

Assume there is a probability space (ft, A, P) on which random variables X[ , X' 2 , . . . and Xy X 2 ^. . . 
are defined in such a way that both of the sequences consist of independent random variables and Xj = X'j = X", 
j = 1,2,.... We identify each of these sequences with a random walk on the integers: let W' = (Wq, W[, . . .) 
be a random walk that starts form at the initial moment and has consecutive step sizes X[, X' 2 , . . ., that is 

n 

I^ = 0and<=^X;.,n = l,2,..., 

3=1 

and let W" = (Wq , W" , . . .) be a random walk that starts form 1 at the initial moment and has consecutive 
step sizes X'{, X 2 , . . ., that is 

n 

Wq = 1 and W'l = 1 + X h n=l,2,.... 

i=i 

Consider the random time 

T = inf{fc : W' k = W^'} 

when the random walks W' and W" first meet. (We use the convention that the infimum of the empty set 
is infinity.) Put W" = (Wtf , W{',...), where 



WH, k<T, 
WL k > T, 



for each k G N. It is evident that W" = W", particularly W^ = W^. Since (W^,W^) is a coupling of 
(W n , W n + 1), by the coupling inequality we have 

d T v(W n , W n + 1) < P(W' n £ W[[) = P(T > n). (2.17) 

We see that if P(T < oo) = 1, then diw{W n , W n + 1) — ^ asn-> oo, and calculating P(T > n) yields a 
bound for the rate of convergence. 

It is very important to note that we did not impose any condition on the relation between the random 
walks W' and W" . They can be independent, but they can also have any kind of dependence structure, 
we only required them to have the same step size distributions. Usually the goal is to define for each 
j = 1,2,... the joint distribution of the step sizes X'j and X" in a way that ensures the finiteness and 
possibly the minimality of T. In other words, one would like to construct couplings of the pairs (XpX"), 
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j = 1,2,..., which guarantee that the random walks W' and W" should meet soon, and therefore that 
P(T > n) is small. One of the ways to do this is given by the so-called Mineka coupling ([20] p. 44), which 
we now define. 

Fix an arbitrary j £ {1, 2, . . .}. Set pj = P(Xj = i), i £ Z. We define the distribution of the steps 
(X'^X'/) inZ 2 by 

P{(X'j,XV) = 1)) = iminfe-i-!,^, J, 

P((Xj,Xj') = = p jti - i minfe.i-i.p^i} - i min{ Pj - ^ 

Thus the couplings force the two random walks to run at most distance 1 apart, in fact, {Sk '■= W' k — W k }k^n 
defines a symmetric random walk, that starts from —1 at time 0, and at each step either stays in place or 
increases or decreases by 1. 

We only calculate the bound for P(T > n) resulting form the Mineka coupling in the case when the 
Xj, j = 1, 2, . . ., are iid random variables with discrete uniform distribution on {1, 2, . . . , L} for some integer 
L > 2, that is pj t i = j;, i = 1, 2, . . . , L, j = 1, 2, In this case 



P(Xj -X'! = 1) = P(X< -X'! = -l) = - Y. ^{Pj,i-i,Pj,i} 



L - 1 



2 ^ 2L 

iGZ 



and 



P(Xj - X'! = 0) = 1 - Y, ™™{Pi,i-uPj,i} = j- 



Using the properties of {Sfcj-fcgN, namely that it evolves by unit steps, that the reflection principle can be 
applied to it, and that S n has symmetric distribution around —1, we obtain 

P(T < n) = P( max S k > 0) 

0<A;<n 

= P( max S k > 0, S n = 0) + P( max S k > 0, S n < 0) + P( max S k > 0, S n > 0) 

0<fe<n 0<fe<n 0<fe<n 

= P( max S k > 0, S n = 0) + 2P( max S k > 0, S n > 0) 

0<fe<n 0<fc<n 

= P(S n = 0) + 2P(5„ > 0) 

= P(S n = 0) + P(S n > 0) + P(5„ < -2) 

= 1 - P(5„ = -1). 

Thus P(T > ?i) < maxigz Pl-S'n = «), and by Lemma 4.7 of Barbour and Xia [7], we have 

t,/ s 1 / . f £- 1 n i 

P(T > n) < — I nmin 



2 V I i 2(7 ^ 



It follows by (2.17) that if W n is a sum of n independent uniformly distributed random variables on 
{1,2,..., L}, then 

d TV (W n ,W n + l) < -4=. (2.18) 
V 2n 

We shall see in Section 6.1 that this inequality can be improved. 
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Chapter 3 

Gumbel-like approximation 



3.1 Preliminaries and results 

In this chapter we are interested in the case of the coupon collector's problem when n > m + 1 is large 
compared to to, so we fix a non-negative integer to. and we shall look at the asymptotic behavior of the 
distribution function 



F n>m {x) :=P( -W n , m - 1<A, : 

\ fc=m+l / 



(3.1) 



as n — > oo. 

As mentioned in the introduction, in 1961 Erdos and Renyi [12] proved for the case m = 0, a full 
collection, that the limiting distribution is the Gumbel extreme value distribution, shifted by Euler's constant 
7 = lim^oo (J2l =1 \ - logn) = 0, 577215 . . ., so that 

hm F n0 (x) = F Q (x) := e" e ( " +T) , x G R. 

For an arbitrary non-negative integer to, this beautiful result was extended by Baum and Billingsley [8] 
shortly thereafter, who proved that 

lim F n , n {x) = F m (x) := -1 f e -^ + i)(y+c m ) e -e~^ c ^ dy 



m 
1 

rn 



— oo 
x+C m 

(m+l)y 



where C m := 7 — Ylh=i h- Much later Csorgo [11] refined this general result, proving 



k 

that the rate of convergence in it is surprisingly fast, namely 



sup \F n . m (x) - F m (x)\ < D m (3.2) 

xes. n 

for some constant D m > depending only on to. 

In this thesis, for every m we give a one-term asymptotic expansion F m (-) + G n m (-) that approximates 
F n ,m{') with the uniform order of 1/n such that the explicit sequence of functions G nm (-) has the uniform 
order of (logn)/n. In particular, it follows that the rate of convergence in (3.2) can not be improved. 

To introduce G n , m (-), consider the density function of the limiting distribution: 

f m ( X ) := F' m {x) = -1 e -e~^ C ™\-(m+l)(x + C m ) = J_ ^^(x)^!^ x £ R 

to! to! 
where e m (x) := c~( x+Cm \ whose second derivative by simple calculation is 
.C(x) = f m (x) [e 2 m (x) - (2m + 3)e m (x) + (to + l) 2 ] 

= £2^1 [ e -+3 (a; ) _ (2m + i)eZ +2 {x) + (to + l) 2 eZ +1 {x)] (3.3) 
to! l j 
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for all x G R. For every k G N, consider also the density function 

, , . f ke~ kx , if x > 0; 
^) := \o, ifx<0 

of the exponential distribution with mean 1/fc, and the convolution 



[fm* h k](x) = / fm( x - V) h k{y) dy= I h k (x - y)f'M) dy, x G 
Then, for n > m + 2, our basic sequence of functions will be 

1 1 /" x 

G n ,m(x) = -— T [f>h k ]{u)du, XER. 

12 k=m+l ^ _0 ° 

It is natural to consider the following version of the Baum-Billingsley theorem: 



(3.4) 



(3.5) 



(3.6) 



F* t (x) := lira F* m (x) where F* m (x) := P ( - W, hm - log n < x 



so that, clearly, 



nJx- f^-logn + f^Y 

\ Lfe=l J k=l / 



x G 



for all n > m + 2, and hence 

F* t (x) = F m i^x - 7 + lj = F ™( x - C ™)> x e 
For every n > m + 2, the corresponding version of the function in (3.6) is 



G n m ( x ) — ^*n,m| % 



1 

£--logn 

k=l 



til 



X G 



k=l 



With all asymptotic relations meant throughout as n — > oo unless otherwise specified, our main result is the 
following 



Theorem 3.1.1 For every fixed m G {0, 1,2,.. .}, 



sup 



\F n>m {x) - [F m (x) + G n>m {x))\ = o(^J. 



(3.7) 



for the functions G ra ,m(') given in (3.6), for which there exist a constant K m > 0, a point x m el, a positive 
function c TO (-) and a threshold function n m {-) € N, all depending only on m, such that 



sup 



i I log n 

\G n , m (x)\ < K m , n>m + 2, 

n 



but 



log n 

\G n ,m{x) \ > c m (x) for all x G (-oo,a; m ), 

1 n 

whenever n > n m (x) . Furthermore, 



sup 



(3.8) 
(3.9) 

(3.10) 



where the sequence {G* m (-)}^ Tn+1 of functions has the same properties as the sequence {G„ i?T1 (-)}^L m+1 
in the first statement. 
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We finish this section by examining the optimality of the results of Theorem 3.1.1. 

It is easy to give an argument showing that any sequence of discrete probability laws corresponding 
to some random variables X n with finite second moments, can not be approximated with an absolutely 
continuous distribution in Kolmogorov distance with an error order that is smaller than l/d n , where d n = 
#{x £ [— 2y/VarX^\ : P(X n = x) > 0}. In order to prove this, we may assume without loss 

of generality that ~E(X n ) = for each n £ N. For a fixed n £ N, by Chebisev's inequality, P(|Jf„| > 
2^/VarX n ) < 1/4, which implies that X n maps into the interval [— 2^/VarX^, 2^/VarX^] with probability 
at least 3/4, and thus there exists a point x n £ [— 2\/VarX n , 2 v / VarX„] for which P(X n = x n ) > 3/(4d„). 
This means that the distribution function of X n has a jump at least 3/(4d„) big at x n , and hence can not 
be approximated at that point with a continuous function any better than 3/(8d„). 

Now, the distribution function F nm defined in (3.1) corresponds to the discrete random variable 
(W njm — ^ n )/n for which d n is of order n, because one can calculate that <j n ~ c m n with some c rn constant 
depending only on our fixed m. It follows that the supremum distance of F n ^ m to any continuous function, 
in particular F m + G n>m , can not decrease in a faster order than 1/n, as n — > oo. This not only proves that 
the error order in (3.7) is sharp, but also that no longer asymptotic expansion of F n ^ m than the one given 
by (3.7) can improve the current error order 1/n. 

3.2 Proofs 

We shall now verify the theorem above. Before embarking on the proof of (3.7), we analyze the function 
G n ,m(') defined in (3.6), to show in particular that this formula makes sense, and prove its properties stated 
in (3.8) and (3.9). We begin with claiming that for every I £ N, 

/x poo l — l 3 ( \ 

e- e ™Wel(u)du= e-V-M^e-^^l)!^^, ieK. (3.11) 
-oo J e m (x) j_Q 3- 

for the functions e m (x) = e~^ +c ' m - ) in (3.3). Indeed, this is true for I = 1, and since 

poo 

e" V dv = e~ e ^et{x) + k e" V" 1 dv, 

it follows for I = k + 1 if it holds for k £ N. So, (3.11) follows by induction. Also, 

DO 

e- em(x) el(i) dx = T{1) = (I - 1)! for all I £ N, (3.12) 

oo 

and we see from (3.3) that /^(-) is integrable on K; in fact, /^/^(a;) dx = 0. 

As is well known, the convolution of two integrable functions is integrable. Since for our convolution, 
from (3.5), 

[f^h k }(x) = ke- kx T e k yf^(y)dy, x £ M, (3.13) 



we have | [/^ * /i fc ] (x) | < k f*^ \fm(v) \ ^Vi ^ s integrability follows directly by (3.3) and (3.11). The last 
inequality also implies that lirn r _>._ 00 [/^* h^x) = 0; in fact, since lim^i^^ /^(x) = directly from (3.3), 
using the dominated convergence theorem in the first formula in (3.5) we also see that lim :r _ i . 00 (x) = 0. 

The first two of the last three properties already make (3.6) meaningful, so that, substituting (3.13) into 
that formula, for the derivative at each x £ R we get 



-, n— l , , n-l r „ x 

G' n , m (x) = -- k [f ^ hk]{x) = ~2^ £ Y~ kx J J kv Civ)dv 



(3.14) 



/c=m+l k— m+1 

Next we note that the derivative of the function in (3.11), 

(c—(^ (.*))' = e"-M [e%\x) - le l m {x)} , (3.15) 



14 



is zero at xq := — log I — C m , is positive for x < xq and negative for x > xq. Thus 



max < e 



_ p -e m (x ) I 



/ £ N. 



(3.16) 



We also see from (3.15) and (3.12) that in fact the j-th derivative fm{-) of f m (-) is integrable on R 
fm\x) = for every j £ {0} U N, not just for j = 0, 1, 2, and hence, as an extension of (3.13), 
the convolutions 

[f$*h k ] (x) = k c~ kx f e^/W (y) dj/, x g R, 



and lim^i^oo f 



make sense as integrable functions for all j € {0} U N and fceE 
Proof of (3.8). To this end, with (3.6) in mind, by (3.13) we have 



[&*h k ](u)du 



< 



k e" 



k e 



- k a 



d// 



du 



< 



< 



felt J I . 



\f;»\- 

\fm(u)\du + 



C{v) dv}du 



\C(v)\dv 



for all x £ R, regardless of what k £ N is. Starting from (3.3) and using (3.15), it is clear that \fm(v)\ in the 
inner integral of the second term is bounded by a linear combination of functions of the form e~ e " l ^e^ n (v), 
in which all the exponents I and all the coefficients depend only on m. Hence, after an application of 
(3.11), that inner integral itself is bounded by a similar linear combination, in the variable u, that has the 
same property. Thus, by another application of (3.11), the sum of the two terms is bounded by a linear 
combination of functions of the form e~ em ^ x 'e^ H (x), in which both the coefficients and all the exponents j 
depend only on m. But all these functions are bounded by (3.16), and hence | f*^ ^fc] ( u ) d«| 5= 2K m /3 
for all x £ R and k £ N, for some constant K m > depending only on m. Therefore, substituting this into 
the obvious term- wise bound for (3.6), 



sup |G„, m (x)| < 



for all n > m + 2 > 2, which is (3i 



in ^ k ~ in 

k— m+1 







1 + 






Jl x 



< K, 



log n 



(3.17) 



Before attending to the proof of (3.9), we note that replacing and /™ by f$ and /m^ ±; , respec 



tively, the argument in the proof of (3.8) above gives 

/ | [fm * h k] («) | d « < %m for a11 x £ R and k £ N, 

J — OO 

for every j G {0} U N, where the constant Km > depends only on m and j. 



(3.18) 



Proof of (3.9). Examining the behavior of given in (3.3), we see that it first increases from = 

f!^(— oo) on a half-line and eventually reaches = /^(oo). Thus the smallest value x m £ R where /^(-) 
has a local maximum is well defined. Consider any x in the half-line (— oo,x m ]. Then the convolution 
[fm*hk](x), given in (3.13) is positive since the integrand is positive on (— oo, x). Thus the fist two displayed 
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lines in the proof of (3.8) above become 
[fm*h k ](u) du = 



/m(«)d«- 



e kv f"(v)dv 



du 



f'^u)du--l [f£*h k ](u)du 



> 



f'm( u ) du 



(3) 



where the inequality is by (3.18) and the first term of the lower bound is positive. Hence, still for the same 

x e (-oo,x m ], 

1 n ~ 1 1 f x 
|G„, m (a;)| = — 22 T [f™*h k ](u)du 

71 H=m+1 00 

/x n-1 n-l 1 

£(«)d« E r-^ 3) E t5 

. "°° fc=m+l k=m+l 



1 

2n 
1 

2n 

logn 
n 

log n 
n 



C(u)du 1 



fc=i 

log(m + 1) 1 7r 2 i4? } 



log n 



12 log n 



.(*)■ 



Since lim.n_j.oo c n , m {x) = 2c m (x), where c m (a;) = j Jl 00 fm( u ) du > 0, there exists a threshold n m (x) £ N 
such that |G n , m (x)| > c m (x) (log n)/n whenever n > n m (x), which is the statement in (3.9). 

We need one more preliminary remark for later use. Noticing that 



2L m /3 := max \f'^{v)\ < oo 



by (3.16), it is that 



i i i L fc=m+l 



k=m+l 

for all n > m + 2 > 2, which is obtained from (3.14) as in (3.17). 



lop 



(3.19) 



Proof of (3.7). The proof being Fourier-analytic, the functions G n . m (-) are first identified in terms of 
Fourier transforms. We approximate the characteristic function tpnm(') = '-Pn.m (•) of F nttn (-) by the sum of 
the characteristic function <p m ( ) of the limiting distribution F m (-) and a "correcting" function. We achieve 
this in four steps. Starting from (p nm (-), we define the functions ip nmi i(-), tp nm ,2(')' Vw,3(-) an d ^nm,_(-)> 
where each of these functions is an estimate of the preceding one — each time obtained by keeping only 
some leading terms from the series expansion of an ingredient — , and Lp nm {-) m tp n m,4(') = fm(') + ipnm(') 
holds for some function ijj nm (-)- For t£K, the approximations yield the error functions 



Pnm,l(t) 
Pnm,2\t) 
Prim, 3(0 
Pnm,4{t) 



(t) - Vw,i(i), 



(3.20) 



and Gn.m(') will be the function whose Fourier-Stieltjes transform is ip nm {- 
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IS 



First, since the characteristic function of the geometric distribution with success probability p G (0, 1) 
[1 + | {e- i4 - 1}) , t e R, where i is the imaginary unit, 

f°° ( { \l ™ 1 

<Pnm(t) = cp„, m (t) := / e itx dF njn (x) = E cxp< it —W n (m) — V - 

J -°° V I l U k= m +i 



by (1.1). Also, for all t € R the limiting characteristic function is 



-it/A; 



exp< it 



Vm(*) := / c itx dF m {x) =El exp 

oo —it/k ( 00 

= n iT¥ =ex Pi e 

fc=m+l A; l,fc=m+l 



Rf>-)]}) 



log 1 



(3.21) 



which follows from the observation of Baum and Billingsley [8] that 



\ k=m+l ^ ' J 



itself suggested by (1.1), where the Y m+ i, Y m+ 2, ■ ■ ■ are independent random variables such that Yj, has the 
exponential distribution with mean 1/fc, and hence the characteristic function E(e ltYfc ) =l/(l-f),t£R. 
Setting 



A nm (t) = expl ^ 
Lfe=m+1 

and noticing that for every i£R, 



log 1 



t e 



(3.22) 



„ fe=m+l 



^ m (t)=exp J2 -T- l0 4 1 + T^ it/n -^ 



{ ^ 1 + -(e _i '/ n - 1) 
A nm (t)exp^- log ^731 



fc— rn+1 



n-l 1 + n 



= A mn (t)exp<- ^ log- 



^it , l nt> 

n T 2 InJ 



Eoo / —it \ ' I 
1=3 I n /! 



1 



fe=m+l 

for |t| < n we introduce the first sequence of intermediate approximative functions 

n-l i 



Vw,iW : = A nm (t)expi- ^2 l°g- 



A- 



— + - (-) 2 

n T 2\ni 



k—m+1 
n-l 



A„ m (i)exp<- ]P log I 1 



k k—m+1 

{n-l 
- 
fe=m+l 



1-f 



(ii) 2 1 



2n fc — it 



(it) 2 1 
2n A: -it 



E 

Z=2 



(-l)'-i /(it) 2 1 



2n A; it 



where the expansion of the logarithm is justified because the inequality 



(it) 2 1 



2n k — it 



t 2 1 



2^ y/k 2 + t 2 
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< 1 



(3.23) 



holds whenever \t\ < n. For all such t, the second intermediate sequence is 

IpnmMt) := A nm (t) CXpl — ^ T _ 71 f 
I fe=m+l J 

+ £ E ^+E^(£ E kh)\ 

^ fc=m+l Z=2 \ fc=m+l / J 



while the third and the fourth are 



f t 2 ™ _1 

k—m-\-l 



and 



4m,4(*) := cxp^ E 

Lfe=m+1 

{oo 
E 
k=m+l 



log 1 



-v-log(l 



+2 »-l 



7 1 — .L 

E i- _ w 



2?i k — it 

k— m+1 



^ n-1 
^ On E 



(it) 2 fc 

2n A: k — it 

k— m+1 



and we notice from (3.21) that ?pnm,4(t) = fm{i) + V'nm(^) for all t £ (— n, n), where 

1 " _1 1 fc 

k=m+l 



(3.24) 



Here fc/(fc — it) is the characteristic function of the exponential distribution with mean 1/fc, so that 



fc - it 



e itx h k {x) dx, te 



where hk(-) is the density function in (3.4). Also, since by (3.50) below the function t H> Pip m (t) is integrable 
on K for every j € N, we can differentiate the density inversion formula twice to obtain / m (-) of (3.3) as the 
inverse Fourier transform 

1 r°° 

= 7^ / e- ltx {it)\ m {t) df, xei 
Since / m (-) is also integrable, as established at (2.2), this can be inverted to get 

/oo 
j tx C(x)dx, ten. 
-oo 

The two Fourier transforms then combine to give 

— — (ii)Vm(t) = / e ite [/£* fc fc ](i) dx, t g R, 
for the integrable convolution in (3.13). Therefore, by (3.14) we recognize (3.24) as 



4>nm{t) 



Atx i 



- X 
(■ X 



n-1 \ 

Yn E t&MWf 1 



/OO />00 
e ite G; m (.T)da;= / c ite dG„, m (x), teR, 
-oo J —oo 



for the integrable function G/ lm (-) for which lim^i^^ G^ m (x) = 0, so that the function G„ ;m (-) is of 
bounded variation on the whole line M. 
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Then the deviation A n := sup,,. gR \F nm (x) — [F m (x) + G nm (x)]\ in (3.7) may be estimated through 
Esseen's inequality (see Section 2.2.), which we use in the form 



A„ < 



2tt 



(t) - [ip m (t) + 4> nm {t)} 



dt + c b 



SU Px£R \fm( x ) + G' m (x)\ 



where b > 1 is arbitrary and ct > is a constant depending only on b and, due to the restriction of the 
arguments t of the intermediate functions ip n m.j(t), j — 1, 2, 3, to (— n, n), the constant c is taken from the 
interval (0, 1). Since max ie t/ m (a;) = ((m + l)/c) m+1 /ro! by (3.16), we see by (3.19) that the second term 
here is 0(l/n). Thus the proof of (3.7) reduces to demonstrating that the same holds for the first term as 
well. This will be split in four parts according to (3.20): we have 



(t) - [cpm(t) + 1p nm (t)] 



dt ^ ^ ^ Rnm,j 
3=1 



and, introducing the set H nc = [— cn, —1) U (1, en], it suffices to show that 



Rr, 



Pnm,j (^) 



dt = 



i(t) 



dt 



tin 



Pnm,j (^) 



dt 



I 3 ni + Ikl = 0{-) for each j = 1,2,3,4. 



(3.25) 



We fix c G (0, 1), let n > 1/c and, unless otherwise stated, assume in all formulae containing the variable t 
throughout that t G [—cn,cn]. 



The case of R nm ,i- By (3.20) and the definitions between (3.22) and (3.24), 



\Pnm,l(t)\ = \tPnm(t) ~ VW,l(*)| 

( 11-1 r 
= cxp< ^2 -^t-IokII- 



exp 



k— m+1 
( n-1 

E 

k—m-\-l 



k 



it , 
T-log 1-r 



- E lo s 

fc=m+l 
n-1 



1-f 



E 



1 _ « -L (») 

1 fc ' 2nfc 
1-f 



The inequality 



|e z -e w |< -{le'l + le^Olz-ttfl, z,»eC, 



where C denotes the complex plane, yields 



|/W(*)I ^ ^ {bnm(*)| + |^nm,l(*)|} *&L(*), 



where 



E lo s 



1 + » ( e -it/ n _ !) 



1 



if , («) 2 



n-1 

logz„ fe (t) 

h— m+1 



fc=m+l x fc ^ 2nfc 

We give upper bounds for each of the functions on the right-hand side of (3.27) 
As usual, let 3? z denote the real part of z G C. Clearly, 



(3.26) 



(3.27) 



n-1 



n-1 



S l nk(t)< E |l°g**(*)|= E 



— m + 1 



loe 



1 



(t) 



(3.28) 
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First we show that $tz n k{t) > 1/2, so that l/z n k(t) is an inner point of the circle of center (1,0) and radius 
1 in C, and hence its logarithm can be expanded about the point (1,0). We have 

1 (1 + f cos £) - i£ sin 1 



1 + £cos 



1 



2nk) k 



nt sin 1 1 



2nfc J 



cos - + 2|S sin f 



J fe A: 2 } K ^ VJO n ^ ^ k 2 ° 11L n ^ k 4n 2 k 2 



'px sin - — tt cos — ) + (^rr sin - — -r^rnz 

<k 2 n k z nJ \2k A n 4n z k 2 f 



sin - + 2^ cos ■ 



v 2k 2 n 



i 



2nk) ' k 2 



This is an even function of t, so we can assume that t > 0. The denominator is obviously positive, and we 
are going to show that each of the three terms in the numerator is non-negative. Beginning with the first 
term, we see that 



tn t t 2 t M 

— sin — cos - = — <! n > 

n k z n k z • L — ' 



i-iy ft 



1=0 



(21 + 1)! \n 



(-i)' ft 



1=0 



{21)1 \n 



t 

Y- 

t 

k 2 
t 



1=1 

oo 

E 

l is odd, I — 1 



i 



1 \ i 



(2/ + 1)! (2Z)!/ n 



4-21 + 1 



( 1 



2/+1 



V(2Z)! (2Z + 1)! / n 21 \{2l + 2)l (2Z + 3)! 



21+3 



,21+2 



( 1 



1 



1 



1 



" fe2 n S oS*=A( 2 0! (2i + l)l (2Z + 2)! ' (2Z + 3)iy n« 



22 + 1 



£ 

k 2 



/ j —- o, > 0, 



l is odd, 1=1 



(21 + 3)! 



where the inequality is by — < c < 1 . Concerning the second term, we note that sin x > ^- if < x < 1. 
Therefore, since - < 1, we have 



' 4 i 2 

> 



tn . t t 

2k 2 Sm n ~ AY 2 k 2 '- 2k 2 2n 2 4n 2 k 2 ~ 4fc 2 



t 2 / * 



> 0. 



Finally, the third term can be settled using t < n and the inequalities sin x > x — ^- and cos x > 1 — ^- 
both valid if < x < 1. Indeed, 



tn t n t n tn ( t t 
—j sm - + 2- cos 2- + 1 > — j 7 

2kt n k n k 2k z \n 6n 6 



2v 1- 



2n 5 



f 4 



2fc 2 12fc 2 n 2 /en 



^- + 1 



> 



f 



2 ^ 



2fc 2 12fc 2 fc 
Returning now to (3.28), we can expand the logarithm: 



5t 2 - \2kt + l2k 2 
12fc2 



> 0. 



n-X 

e«< e 

k—m-\-l 



E 



(-I) 3 '" 1 



1 



Znfe(i) 



- 1 



n— 1 oo 1 

; E E7 

k=m+l j=l J 



Znk(t) - 1 



7 *nk(t) 



(3.29) 
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The inequalities 



E 



< — ugl, and 1 — cos a; > — ^ x 2 , < x < — , 
3! tt z 2 



give the bound 



Znk(t) — 1 


n 


Znk(t) 


= k 



e -it/n _ ! + i* _ fit£ 



l*l f 



1 + f (e- i4 /« - 1) 
1 



< 



\t\ 3 

(ill A 



fc l(l-t + fcosl)-ifsini| 
\t\ 3 1 



< 



&kn2 ^1 + 2^(1-1) (1-cosM) " 6*»' ^ 1 + £n ( n_ l} £ 

J*ll 

6n 2 



1 



1 



which, for any k £ {m + 1, m + 2, . . . ,n — 1} and |t| < n, is not greater than 



\t\ : ' 



i 



6n 2 



< 



|t|3 1 



^ 2 + ^(l-^) 

Substituting these bounds into (3.29), we easily obtain 



< IS E 



t 2 7T 1 



A;— m+1 



fc2 , 8£ (1 _ J,. 



6rt 2 /st 2 1 n 2 12 "3 



Ml 

6 v ; 2 



i-i' 



vv^ 2 +S(i-f), 



< 



" " 1 

£ 4 



6n 2 Jk 2 

k=m+l V ^ 



« , X i-1 

3 



24 n 2 



n-l 



1 

E 



fc— 771+1 



so that by (3.17), 



$L(*) < 



|i| 3 logn 



Next we consider |(^ mn (i)| in (3.27). Since 3? logz = log \z\, z £ C \ {0}, 

r n-l 



|</>nm(*)| 



exp<^ £ 

lfc=m+l 



cxp< - ^ log 



exp< { ~o E log 



i + j (.-■/■ -i 



k— m+1 



71 — fc , 

1 + 2n — —5 — 1 — cos — 

k \ n 



As < \t\/n < c < 1 < 7r/2, it follows from the second inequality in (3.30) that 

8t 2 n-k' N 



(t) < exp 



fc=m+l v 



nk 2 



The terms in the sum are positive, so for n > 2(m + 1) the exponent is bounded by 

L"/ 2 J r OJ 2 nl i r n/2 



1 



E J°8 

A:— m+1 



1 



8t 2 n - f 
7r 2 nfc 2 



< 



1 



lo.c 



m+1 



1 



4f 
n 2 y 2 



dv =■ I 1 ' 



(3.30) 



(3.31) 



(3.32) 



21 



where |_ - J denotes integer part, because the terms in the last sum decrease as k increases. Substituting in 
the integral x = 4i 2 /(7r 2 y 2 ), so that y = 2i/(7r v / x), 



7 i,o 



\t\ rij^TT^) 2 i g(i + x ) 



2tt 

JA 
2tt 

m + 1 
2 

2\t\ 



m 2 

21og(l + x) 



,3/2 



d.r 



; (( m +l),r) |f| /-((m+iv) 



log^l + 
arctan 



4i 2 



(m + l) 2 7r 2 
2|*| 



- I log[l 



7T J(|t) 2 V^(! + X ) 

16i 2 



da; 



41*1 

— arctan 

(m + 1J7T 717T 



Summarizing, at this stage we have 



|vW*)| < |*T 



1 + 1 Vt 2 1 (m+l) 2 7r 2 



1 



16t 2 \I 



exp< 



2\t\ 



arctan 



M (_i 2 

I, 8« 2 ~ 
1 T (m+l)n7r 2 



Using |t| < n and assuming |i| > 1, we can further simplify this to obtain 



Wn m {t)\ <\t\ 



1 



m+1 V (ro+l) 2 7r 

fl 



16 U 
2 » 2 J 



exp< 



01*1 

aiciaii ZTT\ 

7T 1 J 



(m+l)ir 2 



If n, > 2 (to + 1), then the arctan expression in the exponent is a monotone increasing function of \t\ because 
the numerator of the derivative d{as/(l + bs)}/ds is equal to a for any real constants a and b. Hence for 
n > 2 (to +1), which was already assumed above anyway to get to the previous bound, 



|Vnm(*)| < dr, 



\m+l „-r„ m |t| 



1 < \t\ < n, 



where 



dn,m 



_i n(to+1)/2 

(ro+l) 2 7r 2 / 

'1 i 16 W 4 



(m + l) 2 7r 2 



(m+l)/2 



=: cL 



and 



arctan 



2 1 l _ 2} 
tt V m+1 n ) 



2 2tt 

— arctan — — = 

7T (TO + lj7T^ + 



1 l 2 

1 ^ (m+l)7r 2 

Finally, for (3.27), simplifying the second line of the definition of ■0 nm; i(t), 



(3.33) 



h/w,i(t) 



cxp< 



exp< 



exp< 



E 

k fc=m+l 
n-1 



--log 1-- 



ii (it)" 



E 



fc=m+l 
n-1 



1 - 



k 2nk 
it (it) 2 " 



fc 2nfc 



, / t 2 t 4 t 2 



k— m+1 



nk An 2 k 2 k 2 



A;— m+1 



,n — k 



[ l A, / « 

<exp -- 2 logfl + - 

I, fe=m+l v 



nk 2 

8t 2 n - k 
2 nk 2 
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The artificial factor 8/tt 2 < 1 was sneaked in just to get the exact same upper bound as in (3.32) for 
|VVim,i(£)|; an d hence to conclude without any extra work that 



|VW(*)| < d nm \t\ m+1 e- r ™W, l<\t\< 



(3.34) 



for n > 2(m + 1), as in (3.33), with the same d nm — > d m and r nm — > r m > 0. 

Now, recalling the definition of R nm .i = I^m + Inm m (3-25), suppose that n > max{2(m + 1), 1/c}. 
Since \(p nm (t)\ < 1 and \ip nm ,i(t)\ < 1, the inequalities (3.27) and (3.31) yield I* TO < 2(logri)/(3n 2 ). For the 
other term (3.27), (3.31), (3.33), (3.34), and the fact that the functions involved are even, imply that 



jl,2 < 2 ^g" 



t m+3 e~ r " mt dt, 



so that the case j = 1 in (3.25) holds true; in fact, R nm ,i = 0((logn)/n 2 ). 

The case of R nm ,2- By (3.20) and the formulae between (3.22) and (3.24), 



|Pn TO ,2(*)| = \lpnm,l(t) ~ 1^ nm ,2(t)\ 

e M E 



---log 1-- 



k k—m-\-l 
( n-l 



E M 1 



(it) 2 1 



ex pi E 

{k=m+l 



-log 1 - 



k— m+1 

l2 »-! 



2n k — it 



t- 

2n 



T — 



k— m+1 



The inequality (3.26) now gives 



|/W(£)| < \ {|V»»m,l(*)l + |VW, 2 (*)I} 



as an analogue of (3.27), where 



E M 1 

A;=m+1 

E 



(a) 2 i 



k— m+1 



E- 



2n k — it 

i-iy- 1 ( m 2 



n—l ^ 

E u _ 



2n ^— ' k — it 

fc=rn+l 



3=1 



(a) 2 1 

2ro fc — it 



< 



E E 

fc=m+l j=2 

71—1 OO 1 

E E7 

k=m+l j=2 ■' 
n-l 



/ (it) 2 



j \2ra(fc-if) 



2n(fc - it) 



< 



" x 1 



fc — m + 1 



t- 



2 \2nVfc 2 +t 2 



n—l 00 1 

= E E- 

/ 

E 



k— m+1 j—2 

t 2 



t- 



3=2 



j \2n\/k 2 +t 2 
3-2 



2nVfc 2 +l 2 



Since |i| < n, we find that t 2 /(2nVk 2 + t 2 ) < n 2 j{2n\Jk 2 + n 2 ) < 1/2, and so 



n-l 



%l(t)<\ E 



/c— m+1 



t- 



2nVfc 2 + t 2 



2 00 /1 \J- 2 



3=2 



n-l 



^lo) ~ 2^2 U2 T +2 - 7^2 p 



4n 2 ^ fc 2 + i 2 ~ 4n 2 ^ fc 2 ' 
fc— m+1 fc— 1 



Therefore, 



7T 2 t 4 

nm( ) S 24 n 2 ' 



(3.35) 



(3.36) 
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Next, recalling the notation for A nm (i) from (3.22), we write 



\i>nm,2{t)\ = \A nm (t)\\B nm {t)\ with B„ m (t)=exp< — E . _■, |i (3.37) 

I fe=m+l J 



and estimate the two factors separately. First, 



A nm (t)\=exp\- J2 logl-^ [ =cx p|-^ E lo sf 1 + l2 



ra-l 



2 ^ V ft 2 

fc=m+l 



k fc— m+1 

skp H/„>( i+ ?W' 

where the inequality holds again because the terms of the last sum decrease as k increases. Substituting 
x = t 2 /y 2 , so that y = \t\/y/x, we obtain 

\ t \ fi^TiY l g(l + X ) 



-2 log (1 + 3 



exp< 



(m + iy 2 



1+1 



H^V | t | r (^r) 2 ! 

ax 



■ exp< — |t| arctan 



(±) 2 2 ^(l + x) 

1*1 (m+T — ra, 



1 + 



/ 2 



(m+1) 77 



whence 

/ l i \(™+ 1 )/ 2 

l^-W|<l*l /2 exp| |t| arctan t2 + {m + 1)n 



If |t| > 1, then 



\A nm (t)\ < a nm \t\ m+1 exp<j-|i| arctan M?_ZL_J) j> ( 3 . 38 ) 

f- 2 + (m + l)n 1 



where 

/ 1 \(m+l)/2 x H±i 

_ V 1 + (m+l)'->V . 1 



(1 + J,)"/ 2 V (m + 1) 



and the arctan expression in the exponent, as a function of |f|, is monotone increasing on (0, y (m + l)n) 
and monotone decreasing on (m + Ijn , n) because 

d / s(n — m — 1) \ (n — m — 1) (n(m + 1) — s 2 ) 



ds \s 2 + (m + l)nj {s 2 + {m + l)nf 

Then it follows that 



\A nm {t)\ < f |r +1 exp|-|t| arctan ^g^}, if t g [1 V(m + l) n); 
a »™ " \ |i| m+1 cxp|-|t| arctan , if < G [^(m + l)n , n). 

Next we deal with the other factor in (3.37), for which 



|2W*)|=«p|»(£ E A;H=«p|£ E 



2n ^ fc - it \ \ 2n ^ k 2 + t 2 

< exp<j ^ i 2 1 , whenever \t\ < n, (3.40) 
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by (3.17). This will be good enough for small \t\, and for moderate \t\ it may be written as 

\B nm (t)\ < exp j 3 ^/^+ T | t | jggg j , if \t\<y/(m + l)n. (3.41) 

To obtain yet a third bound, useful for large \t\, note that d{y/(y 2 + i 2 )}/dy = (t 2 — y 2 /(y 2 + t 2 ) 2 . The 
behavior of this derivative shows that for \t\ < m+1 the terms of the sum in the formula of \B nm (t) \ decrease 
as k increases, while for m + 1 < \t\ < n the terms increase until k reaches \t\ and decrease afterward. Hence 



1,21+2-/ „2 , +2 ^ ( 2±*2 e 



1 k 2 +t 2 ~ J m y 2 + t 2 y [\t\\ 2 + t 2 b \m 2 + t 2 [\t\\ 2 +t 2 



HZn? k| V2n 2 3^2n + 2 

<log V — + fT^ < T + ^ = ^^' 

where, by elementary considerations concerning integer parts, the second inequality holds provided y/2/(y/2— 
1) < |£| < n. Thus, substituting this bound into (3.40), 

\B nm (t)\<exJ\t\ 3V ^ +2 |, if ^^-<|t|<n. (3.42) 



Introduce now the sets S nm = [— \J (m + T)n , — l)u(l, (m + l)n] andT„ m = [— cn, —^J (m + l)n )u 



(•y/ (to + l)n , cn] , so that H nc = [—cn, — 1) U (1, cn] = S nm U T„ TO in the second term of R nm ,2 = + X 
in (3.25). Since |i/w,i(t)| < 1 and |A nm (t)| < 1, (3.35), (3.36) and (3.37), combined with (3.40), give 



2.2 

I! I'll 



nm S 48 n 2 i.J 1 



1 +exp<j -t 



3 . 2 lo 



,2l + exp{|^} 
d ^96 i L ^°' 



while by (3.35) and (3.36) only dropping the factor 7r 2 /48 < 1 for simplicity, 



Inm<\ I 1^1^.1(01^+4/ \t\ 3 \^n m . 2 (t)\^+^2 I l*| 3 |^fW7.,2(t)|d* 

_. 72,2,1 | r2,2,2 , j2 
■ ri.m ' nm ' n 



Here I 2 ^ 1 = 0{l/n 2 ) by (3.34). Also, by (3.37), (3.39) and (3.41), 



9 rV ( to + 1 )™ On rV ( m + 1 )» 

2 



< 4 i 3 |Vw, 2 (t)|di < r^e-^dt, 



n 

where a nrn — > a rn and 

n — to — 1 Vto + 1 log n 1 

b nm := arctan — — ; — =■ -> arctan — > 0, 

1 + (to + l)n 2 Y n to + 1 

so that J 2 ; 2 ' 2 = C»(l/n 2 ) as well. Finally by (3.37), (3.39) and (3.42), 

< ^ r t 3 \^ nm , 2 (t)\dt <^f t m+4 e-^dt 
with the where 

n - to - 1 3V2n + 2 it \/2 

Cnm := arctan >• — — > 0.078 . 

n + to + 1 6n 4 2 

Thus 7 2 > 2 > 3 = o(l/n 2 ). So, the case j = 2 of (3.25) holds; in fact R nma = 0{l/n 2 ). 
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The case of R nm ,3- Recalling (3.20) and the formulae after (3.22), we have |p nTOj 3(t)| = \ip n m,2(t) — VWra,3 
|^4nm(£)l \C n m(t)\, where A nm (t) is given in (3.22), and already occurs also in (3.37), and 

00 ^ [ t 2 " _1 1 



l\ 2n ^ k-it 

1=2 L k=m+l 



Clearly, C„ m (0) = 0, and for t ^ 0, 



|C„ m (i)| < — 

1=2 

oo 

<y- 
- ^ u. 

1=2 

oo 



k= m+1 

2 



1 



;=2 



k-it 
1 



d// 



oo 1 

OO 1 



Z=2 



2n ^ 

^ r/i*! i_ 

2^ 7o v^TT 



dx 



2n 



— rsh — 



2n 



-r— rsh( 7-7 



2n 



1*1 



where, for momentary gain of space, rsh(x) := arcsinh(x) = log (x + \/x 2 + 1 ) is the reverse (inverse) to the 
hyperbolic function sinh(x) = (c x — c~ x )/2, Thus, 



(3.43) 



Pnm,a(t)\ < |Aim(<)| ^2 rsn ( TIT ) ex H ~ rsn ( TIT 



1*1 



2n 



Now we can estimate R nm ,3 = Inm + ^nm i n (3-25). 

Concerning the first term, notice first that for the derivatives of the functions v„ (s) — s 3 rsh 2 (n/s) 
and w n (s) = s 2 rsh(n/s)/(2n), for s € (0,1], we have 



v' n (s) = s 2 rsh(-) 3rsh(-) 
which is positive since 3rsh(l) — 2 > 0.64, and 



2n 



Vs 2 + 



> s 2 rsh(-) [3rsh(l) - 2] 



W " (S)= 2^ 



2rsh(- ] - 



V s 2 + n 2 



> l[2rsh(l)-l] > — — >0, 
~ 2n l V ' J 2n 100 



so both w n (") and exp{u>„(-)} are monotone increasing on the interval (0,1). Hence, using (3.18), the fact 
that |A nm (i)| < 1 and the evenness of the functions involved, we can bound the integrand by its value at 1 
to obtain 



J3,l < J_ 
nm - 2n 2 



v n (s) e w "( s) ds< 



; arc S inh(n)/(2n) arcs i n h 2 ( n ) 



The asymptotic equality arcsinh(n) ~ logn then shows that — 0((log n) 2 /n 2 ) . 

For J 3 ; 2 , using (3.43), (3.38) with the a nm — > a m given there, the evenness of the integrand and the 
fact that the function 1 1— > arcsinh(n/i) is decreasing, we get 



r3,2 < <W arcsinh 2 (77,) F 
2 n z J l 



t m+4 e -tg nm (t) d . 



(3.44) 



where, with t € [l,7i) everywhere in what remains of the present case j = 3 of (3.25), 

t(n — m — 1) i 



Jnm(t) = arctan ■ 



±-± arcsinhf 2) . 
1) 2n V</ 



Here 



t 2 + n(m + 1) 
(n - - 1) [n(m + 1) - t 2 ~\ 1 / arcsinh(n/£) 



[n(m+l)+t 2 } 2 + t 2 {n-m-l) 2 2 
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The first term of this expression is positive if t < y/n(m + 1) and negative if t > y n{m + 1), while the 
second term is negative for all t € (0, n), since this term takes on a negative value at t = n and is an 
increasing function of t on this interval: 



1 / arcsinh(n/i) 
2 



1 



y/P + n 2 J 2t{t 2 + n 2 fl 2 



> 0, if < t < n. 



In particular, g nm (-) is a decreasing function on the interval (y/n(m + 1), n), and hence g nm (t) > g nm {n) 
for all t 6 (\Jn{m + 1), n), where this value is 

n — m — 1 arcsinh(l) arcsinh(l) 

9nm\n) = arctan > arctan(l) > 0.34. 

n + m + 1 2 2 

Therefore, for all n sufficiently large, 



t m+i c -t gnm (t) dt < 



•y/ n(m+l) 



y ra(m+l) 



r i+4 e-(°- 3)t dt = o(l). 



(3.45) 



For t G [1, ^/n(m + 1) ) we replace the leading * of the second term in g nm (t) by n(m + 1), and then notice 

sing function of t on the interval [1, i 

t(n - m - 1) \fm + 1 arcsinh( j) 



that the resulting lower bound is an increasing function of t on the interval [1, \J n(m + 1) ). This way we 
obtain 



g nm (t) > arctan 
> arctan 



t 2 + n(m + 1) 2 s/n 

n — m — 1 \Jm + 1 arcsinh(n) 



l + n(m+l) 2 
and this converges to arctan(l/(m + 1)) > 0. Hence for all n large enough, 



y/ n(m+l) 



t m+4 e -t 9nm (t) dt< 



•y/n(m+l) 



1 + 4 e -(l arctan ^j) * ^ 



(3.46) 



Now (3.46), (3.45) and (3.44) together give = 0((l og n) 2 /n 2 ) again. Thus i? nm , 3 = 0((log n) 2 /n 2 ) as 
well, and so the case j = 3 in (3.25) is amply satisfied. 

The case of R nm ,4- Using (3.20)-(3.22) and the formulae above (3.24), we see that \p n m,4(t)\ = 
\ipnm,3(t) - Vw,4(0l = \D nm {t)\ \f m (t) - A nm (t)\, where 



Dnm (*) 



f 2 "- 1 



^ 1 t Z ±- 1 

> < H > — ^= 



< 



r 2 2 ™ i r 
fe=i 1 1 k 



2, if 1*1 < 1; 
21*1, if 1*1 > 1. 



With this factor done, inequality (3.26) gives another analogue of (3.27), namely 

1/wWI < + \A nm (t)\}6\t(t), 



(3.47) 



(3.48) 



where, since the assumption \t\ < cn for some c £ (0, 1) ensures that the logarithms can be expanded for all 
k = n, n + 1, . . ., as noted at (3.23), 



E 



if 



log 1 



it 



oo oo 



< < V ^ 



k=n 3=2 J j=2 J y 



dy 



x ^ \t\ j 1 r^" 1 



„ t 2 n 

H j(j ~ 1) n>~ x (n - I) 1 ' 1 ~n~l 



1 1 

2 + 6 



E 

j=3 



\t\ n 



n n — 1 



3-2 
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Let n be large enough to make n/(n — 1)<(1 + c)/(2c), where c is as above. Then 



\t\ n 1 + c 1 + c 

< c— — = — — < 1, 



and so 



Hence, for all n large enough, 



E 

3=3 



n n — 1 



1*1 g 
n n — 1 



2c 



J-2 



< 



1 _ i+c 
1 2 



S [4] (f) < f i+£ 



1 1 1 + c 



2 6 1 -c 



< 



1 + c 

1 -c 



t 2 (1 + c) 2 
n c(l — c) 



(3.49) 



Next, by classical results on the T function ([22], §8, for example) we see that 



\<Pm(t)\ 



k=l 



fc=i 



-it/fc 



= |r(l-ii)e- it7 | J 



k=l 



k 



= |r(i-it)| 



n(*+S 



-2ir|tj 



: e 2 



ni'+S 



\ fe=i 



starting from (3.21), which implies 



\<Pm(t)\ < 



^\t\(l + t 2 ) m e~ nltl/2 , whenever |t|e[l,oo). 



VI -e- 2 ^ 

Also, since (to + l)n + 1 > n + to + 1, combining the two cases in (3.39) we have 

n — m — 1 



arctan ■ 



1 + (to + l)n 



if |t| G[l, cn], 



(3.50) 



(3.51) 



and we are ready to deal with R nm ,4 = Imn + ^tm m (3-25). 

The inequalities \p m (t)\ < 1 and |A nm (t)| < 1 and (3.47)-(3.49) imply 

1 2(1 + c) 2 f 1 ulJj 2(1 + c) 2 1 



/4,1 < 

mn " n c(l - c) J. 



j m = 



c(l — c) n 



for all n large enough. Also, collecting the five bounds from (3.47)-(3.51), for all sufficiently large n we 
obtain 



^ Qnm 2(1 + c) 2 f TO+ 3 



mn — / -i \ 

n c(l — c) 



cxp 



arctan ■ 



71 — TO — 1 



1 2(l + c) 2 V2^ 
n c(l - c)\/l - e -5 



1 + (to + l)n 



Since a nrn — > a rn and arctan([n — to — 1]/[1 + (to + l)n]) — > arctan(l/(m + 1)) > 0, we conclude that 
Imn = 0(l/n). Therefore, R nm ,4 = 0(l/n), establishing the case j = 4 of (3.25) and thus completing the 
proof of (3.7) in the theorem. 

Proof of (3.10). Let E n = Y^k=i 5 — logn — 7, for which, by a classical asymptotic expansion due to Euler, 



E n 



1 1 



for some n £ (0, 1). 



2n Yin 2 120n 4 

Using the forms of the three ingredients given before the statement of the theorem, for the deviation 
A; := sup xeR \F* m (x) - [F^(x) + G£ jTO (x)]| in question we obtain 

A* t = sup \F n ^ m (x — C m — E n ) — F m (x — C rn ) — G n ,m(. x ~ C m — E n )\ 
= sup \F n>m (y) - F m (y + E n ) - G„ tm (y)\ 

yGR 

< sup \F nm (y) - [F m (y) + G n . m (y)]\ + sup \F m (y) - F m (y + E n )\. 
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Hence (3.7) and the inequality, obtained by the mean value theorem, 

sup \F m (y) - F m (y + E n )\ < \E n \ max f m (y) = 0[ - ) 
v m y£ R \ n J 

yield the desired statement in (3.10). Also, the functions G* m (x) := G n , m (x — C m ) clearly inherit the 
stated order properties of G n ^ m (x), x £ K, since the shift C m is constant. Then so do the functions 

@n,m( x ) = Gn,mi x - E n ), x eR, because 

sup \Gl (x) - G* nm {x)\ <\E n \ sup \G^ m {x) \ = O 
where the last bound is due to the inequality in (3.19). I 
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Chapter 4 

Normal approximation 



In this chapter we prove an error bound for normal approximation to the coupon collector's standardized 
waiting time. We introduce the distribution functions 

As mentioned in the Introduction, Baum and Billingsley [8] showed that if the to goes to infinity along with 
n, but slowly enough to let the sequence (n — m)/y/n tend to infinity as- well, then the standardized W n , m 
is asymptotically normal: 

1 f x 2 

lim F nm (x) = $>(x), where $(x) := —= / e _s / 2 ds, i£l. 

' V27T J-oo 

The following theorem gives a bound for the rate of convergence in this central limit theorem. 
Theorem 4.0.1 For all n > 3 and 1 < m < n — 2, we have 



n 1 

sup \F n<m {x) - *(x) \< C , (4.1) 

m a n 



where C = 9.257. 



One can check that the bound given by Theorem 4.0.1 goes to iff m goes to infinity along with 
n, but slowly enough to let the sequence (n — m)/y/n tend to infinity as-well, which is in accord with the 
central limit theorem stated above. Indeed, this follows easily from the asymptotic formulae for cr 2 , given 
by Baum and Billingsley [8]: 

If S _> d for some d £ (0, 1), so that 2=2 -> 1 - d, then al ~ n 1 ~ d+ . t "° srf . 

n V ' /' n in d 



If ^ 1, so that 2=2» -> 0, and -> oo, then cr 2 . 

n 3 n 7 n 3 71 2 



===^ -> oo, then al - 1^21)1 
EH^o so that 2^™ _> I and m -> oo, then cr 2 - si. 

These asymptotic relations then give the following typical examples: 

If to ~ dn for some < d < 1, then ^<J n ~ constant • -y/n. 
If to ~ n — n a for some | < a < 1, then ^cr n ~ constant • n Q ~^ . 
If to ~ for some < [3 < 1, then — <j„ ~ constant • ti' 3 / 2 . 
If m ~ logn, then ^<r n ~ constant • y/\ogn. 

Proof. We estimate the supremum distance between the distribution function F n ^ m and the limiting dis- 
tribution function $ in terms of their characteristic functions, using Esseen's smoothing inequality. Since 
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the characteristic function of the geometric distribution with success probability p <G (0, 1) is pe 1 '/ (1 — qe 1 *), 
where i is the imaginary unit and q = 1 — p, by (1.1) we have 



(Pn,m (^) 



e ixt dF n , m (x) =E 

n \ 



H^K-J +1 i]}) 



k—m+1 



=k&t/cr„ ' 



while the limiting characteristic function is e lxt d<fr(x) = c~*"/ 2 , isl. Choosing the main parameter in 
Esseen's inequality (see Section 2.2.) to be c n o n , where, with any fixed c S (0, 1), the sequence c„(m) = c n 
is given by 

c(m + 1) 



c n (m) := min< 1, 

\Jn\n — m — 1) J 

the inequality in the case of our distribution functions takes on the following form: 



sup \F n , m (x) - $(x)| < 



2tt 



¥>n,m(*) - e * 2/2 



(If 



Cb 1 
/ ZlT C n (7 n 



(4.2) 



(4.3) 



where b > 1 is arbitrary and Cf, is a positive constant depending only on b. 

Since we restricted the domain of the characteristic functions to (—c n a n ,c n a n ), from now on we 
assume that this interval is the domain of all formulae containing the variable t. We emphasize that by the 
definition of c n in (4.2) this means that, on the one hand, \t\ < <7„, and, on the other hand, for any c € (0, 1) 
chosen in c n and k 6 {m + 1, . . . , n — 1}, 



\t < 



c{m, + 1) 



-O n < 



ck 



--On < 



k 



n{n — tyi — 1 ) yj n(n — k) -J 'n{n — k) 



(4.4) 



We estimate the deviation \(p n . m (t) — e * / 2 | in the integrand on the right-hand side of the inequality 
in (4.3) through the following heuristic steps: 



<Pn,m(t) = exp' 
( n-l 

«expi J2 



■ n-l r 

E 

. k— m+1 



n — k —it 



los 



log 1- 



n — k t 



it/o-„ 



exp< 



> k— m+1 
' n-l 

E 

(k=m+l 

ex Pi E 

lfc=m+l . 
f n-l 

E 

k A;— m+1 

t 2 ' 



n — k —it k 
— h log log 1 

k <7 n n 



- k 



it t 2 
o n 2ol 



n — k —it 
k o n 



log 1 



n— k ( it f 2 



a„ 2a 2 



cxp< 



n — k —it n — k I it i 2 
o„ 2a 2 

2 ;+3 



k a„ 



1 (n - k) 2 / it t 2 

2 k 2 \e~~2a} 



n(n-k)t 2 (n-k) 2 it 3 (n - k) 2 t 4 1 



2fc 2 a 2 2k 2 o* 

n-l 



8k 2 



cxp i 7 cxp E 



(n-kl ( -tf> j4_N 1 



k A;— m+1 



2fc 2 



4(T 4 7 



expi -- 



At each approximation a certain function was replaced with the first few terms of its series expansion. At 
the second one this was done with a logarithmic expression, whose expansion about 1 exists, because for an 
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arbitrary term of the sum in ifn}m('), that is, for an arbitrary k G {m + 1, . . . ,n — 1} 
n — k 



it t z 
<j n 2a 2 



k \t\ 



ck 



1 



1 + 4cr2 < k ^/ n (n - k) V " ' 4n(n - fc) 



by (4.4), which gives 



n — k 



it t 2 
o n 2<7 2 



2n-k 

< c < c < 1. 

2n ~ 



(4.5) 



Now, the errors resulting from the first two of our three approximations can be estimated applying 
the following inequality 

\e^-e^\<U\e^\ + \e^\}\ Zl -z 2 \ 



for arbitrary complex numbers Z\ and Z2- This yields 

\<Pn,m(t) ~ <pW m (t)\ < \{\Vn,m{t)\ + \ <P l n}m (* ) | } €}m (*) . 



(4.6) 



where 



and 



where 



<#L(*) 



n-l r 

E 

k— m+1 



log 1 



n — k 



H /°" -log 1- 



n — k 



it t 2 
a n 2a 2 



(4.7) 



4 2 U) = 



Tl-l 

E 

fc— m+l 



log 1 



-k ( it t 2 



a n 2a 2 n 



n-k ( it t 2 \ l(n-k) 2 ( it t 



k \a n 2a 2 ) 2 k 2 \a n 2a 2 n 

n—1 oo 

E E 



-1 ( n-k \ 3 (it t 2 > 

J V * J Un 2(7,2 



fc— m+1 j— 3 

Summarizing, for the estimation of the integral 

¥>n,m(*)-e-* 2 / 2 



dt 



in (4.3), we use the intermediate approximative functions <^n, m (") and <^n,m( - )) an d the inequalities above 
concerning their differences, obtaining have 



i r™' 7 " 




2 7-c„ CT „ 


t 



i r^- 


V?n,m(*) 


2 7-c„ CT „ 







<Pn}m(t) 




t 



d(*)dt- 



C(f)df + i 



^nL(i) 



€L(t)dt 



di. 



(4.8) 



Now we give upper bounds for each of the functions occurring in the integrals above. 
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First we consider |<^ ra ,m(i)|- By simple computation 

( n-l 



\<Pn,m(t)\ 



exp< 



exp< 



E 

l,fc=m+l 
' n-l 

E 

, k— m+1 



fc —it 



k a n 

log - - log 

n 



log log 1 - 



1 



n — k ■, 



\t/a„ 



= ex P I 



log log 4/1 

n V 



- k 



n — k t 

— 2 cos — 

n a 



n 



fc— m+1 
^ n — 1 



log log 

n 



- fc 



1 — cos 

0V, 



cxp<^-- ^ log 1 + 2 



fc— m+1 



n(n — k) 



1 — cos 

On 



Since t/a n < 1 < ir/2, we can continue with applying the inequality 1 — cos a; > -^ix 2 , true for x £ (0,7r/2), 
and obtain 

8 n(n - k) i 2 

■ 



\<Pn,m(t)\ < cxp<j ~ iogf 1 
fc=m+l ^ 

Now we see from (4.4) that we can use the inequality 

,,2 



X 



\og(l + x)>x- —, if x G (0,1), 



(4.9) 



which yields 



f 1 

\Vn,m{t)\ < exp^ -- ^ 



A;— m+1 



n(n - k) t 2 ( 8 32 ra(ra - fc) t 2 



fc 2 



r 2 7T 4 fc 2 



Using once again the bound in (4.4), on the second t 2 in the expression above, and recalling (1.3), we easily 
get 



20 

\<Pn,m{t)\ < exp< jt 



(4.10) 



Next, in a completely analogous way 

( n-l 



l^mWl 



ex pi E 



^ fc— m+1 

n-l 



k -it 



exp< 



k <r n 
E lo s!- : 



k—m-\-l 



log 




-( 


it 




0"/) 



- fc / it t 2 



cr„ 2cr 2 



2a 2 



ex PS E ^sf 1 

I fc=m+l ^ 



n(n - fc) t 2 (n - fc) 2 t 4 



n-l 



k 2 fJ 2 

n(n — fc) t 2 

LOg ^ i + 

A:— m+1 

Again, (4.4) allows us to use the inequality in (4.9) to obtain 



fc 2 4cr4 



<expi-i E ^f 1 



\<P$ m (t)\ <ex P <j-i J2 



fc— m+1 



n(n — fc) t 2 
fc 2 ^ 



1 - 



i(n- fc) t 2 
2fc 2 Vi 
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and now, another application of (4.4), and recognizing (1.3), yields 



<U*)| <ex P - 



Also, by elementary considerations and (1.3) again, 

( n-l 



>-^-\ ex v\ E 

{ k=m+l 



{n - k) 2 ( -it 3 



exp 



jexpj 

J K k=rn-\ 



- eXP l"Tr XP l E 



m+1 

n-l 



k=m-\-l 

t 2 ' 



(n-fc) 2 t 4 
n(n — k) t A 



ex Pi ~~2 r CX P 



< exp<j -— J-exp-j — 



where the last inequality follows from the assumption that \t\ < a n . Therefore 

[2] '-i - 3t 2 



<LW <exp 



Next, we see for Sn} m (t) in (4.6) that 



n-l 1 _ n-k e it/* n 

y io s \~ 



n-l 



\—k 



E log i+ — 

=m+l y 

n-l / 

E lo § 1 



Ji t 



-fc At/a?, 



-i n—k ( -1 I it t 2 

~ 1 ' oVi 2ctJ 



k=m+l 



Since |e iu - (1 + m - *f )| < ^- for all ueR and \t\ < a 



k ]L i t 2 

n—k a n 2cr 2 



n-l 



^ log(l + z„, 



A;— m+1 



kn.fcWI < 



T ,„4 -r 



Jill i jL 

6<tJ 3 2cr 2 1 

< P== < + 1 - = o < 1. 

t 2 / t 4 / t 4 3 



(n-fc) 2 ^ 4(T 4 -r „_ fc ff 2 

the logarithmic expression can be expanded, so that 



g g(_-ip" +1 < fe W 

fc— m+1 j — 1 



< 



E E 

A;— m+1 j — 1 



kn.fcWP 



n-l 

* E 

fc=m+l 



1 oo 



i=i 
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Using the upper bounds just given for |z ra> fc(t)|, we obtain 

n-l 

e^)< e 



k—m-\-l 

5 Itl 3 



\t\ 3 
60? 



f. 1 



(n-fc) 2 1 4(7* 1 n-fc cr 
n-l 



E 



1 



< 



5 |t| 3 ^ n-fc 



E 



24 (j3 ^ / fc 2 1 t 4 1 n i 2 " ~~ 24 0-3 ^ k 

k=m+l y („_fc)2 1 4cr 4 "I" „_ fc CT 2 k=m+l 



24 o-3 ^ k 2 

" fe=m+l 



so that recalling (1.3) again, 



We now turn to <5n, m (t) in (4.7). Clearly, 



AM: 

24 a n 



(4.13) 



n — 1 oo 

€U*)< E E 

^ n— 1 

<- E 

- 3 ^ 

k— m+l 





-it 


t 2 




cry 




H 2a 2 





(n - fc) 3 
~ P 



-it g 
+ 2^f 



3 I 00 



E 

3=0 



-it 

OV 2(7? 



By (4.5), the infinite sum here is not greater than J^jLo ^ = 1/(1 ~ c )i f° r an y fixed c S (0, 1) chosen in the 
definition of c„, and this, together with the inequality 



-it t 2 3 |t| 3 



1 



cr„ 2cr 2 

which is true because |t| < a n was assumed, gives 

n-l 



t 2 \ 3/2 It 



a 3 V4 



3/2 



5^5 |t| 3 ^ (n-fc) 3 



E 



24(1 - c) 0-3 ^ fc 3 



5\/5 |t| 3 
24(1 - c) <7 3 



For the remaining sum r n , m here, by (1.3) we obtain 



(n — fc) 2 n(n — fc) (n — m — 1) 



hk 



< 



1) . ^ 



n(m + 1) ^— ' fc 2 

v ' k=m+l 



E 

fc— m+l 

(n — m — l) 2 2 n — m — 1 2 ^ \/ n ( n — m — I) 
n(m + 1) 



n(n — fc) 



and we conclude 



4!(*) < 



m +1 m+l 
5\/5 Itl 3 



24(1 - c) c„cr„ 



(4.14) 



35 



It remains to deal with the deviation |<pn' m (t) — e * J / 2 |. We have 



|AW-e- t2/2 | 



where 



n-l 



k—m+1 



= exp 
= exp 

< exp 

< exp 

(n - kf 



exp-j 



^ (n-k) 2 (-it 3 t 4 \) 



E 7 

J'=l J 

9 I ,-| 



n-l 



J'=l 



n(*)E 



E 

fc— m+l 
n-l 

E 

/c— m+l 

S n.m(t) 



n - k) 2 {-it 3 t 4 



k 2 \ 2a r 3 8^ 



(n - k) 2 



2t„ 8a* 



3=0 



J- 



-it 6 t 4 



n-l 

E 



(n - fc) 2 / i 6 t s 



k 2 V 4cr6 64ct8 



We give two different bounds for s n>m (t). First, since \t\ < er„, by (1.3) we can write 



|t|3 / 

Sn,Jt) = ^-tW 1 



f 2 



E 



(n-k) 2 Vl7\t\ 3 ^ n(n-fc) Vl7|tp 



2cr3W 16a 2 ^ A: 2 

" V "fc=m+l 



< 



E 



8(73 £2 
™ fc=m+l 



For the second bound, we first estimate (n — fc)|i|/(ner n ). Applying (4.4), and the fact that the maximum 
of the function J x(l — x) is 1/2 on the interval [0,1], we get 



n — k \t\ n — fc 



n <r n n y/n(n - fc) 
This inequality, along with |t| < a n and (1.3), gives 

a I ~ n-l 



n — fc fc / ?i — fc fc 1 
< W < 

n n 2 



^n,m (^) 



1 



f 2 



E 



2o- 2 V 16ct 2 , 

n V n k=m+l 



n(n — fc) n — fc |t| 

fc 2 71 (T„ 



16 



Continuing, we apply the first bound of s n ^ m (t) to the function before the sum and the second one to each 
term of the sum, to get 



UP] 

rn.m 



W _ e -*7 2 |<xiZM! exp < 



1 a/17 N 



2 16 



t 1 



< 



17 



■ cxp< t' 

16 



8 a- 

Now, collecting the bounds, we can return to the estimation of the integrals in (4.8). We obtain 

20t 2 l 1 v^tt 13 / 2 



(4.15) 



2 

by (4.10) and (4.13); 



by (4.11) and (4.13); 







' —C n <7 n 


t 



1 5 c°° 
S m m (t)dt<-- exp 
cr„ 48 ./_™ 



i 2 dt 



1 /" C " CT " 


¥>n,m(*) 


2 y- c „a„ 


t 



cr„ 3840 



1 

fn 12 ' 



1 r™' 7 " 




2 J-c n <r n 


t 



€L(*)d*< 



1 5^5 
c„cr„ 48(1 - c) 

1 5V^ 
C„<7„ 12(1 - c)' 



cxp^ -- ^ 2 dt 
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by (4.11) and (4.14); 



^ rc„a„ 

2 



fn,m(t) 



1 



5\/5 



c„cr„ 48(1 - c) 

1 5\/3Tbf 
c„cr„ 54(1 - c) : 



3r 

exp<! — — }t z dt 



by (4.12) and (4.14); and finally 



dt < 



i vTf 

t7„ 8 



3t 2 1 2 , 1 4^51^ 

exp< } t dt = 

16 | <r„ 9 



by (4.15). Since l/<7„ < l/(c„<7„) by the definition of c„ in (4.2), substitution of all these bounds into (4. 
yields I n>m < C/(c n cr n ), where 



C 



V5n 13 ' 2 5^ 5a 



t>7T 



3840 12 12(1 - c) 54(1 - c) £ 
Writing this back in (4.3), we obtain the inequality 

(7 

sup |F„. m (x) - < 

iei c n CT n 



5\/307r 4V51tt 2.5503 

— < + 7.3566 



1 - c 



with the constant 



C 



™ + 7.3566U + ^L. 
1 - c J 2tt x/2^ 



According to Csorgo [11], the minimum of c& is 4.439 occurring at 6 = 1.868, and with these values C > 
0.7583/(1 — c) + 3.9581. Also, it is easy to see from the definition of c n in (4.2) that c„ > c^, thus 



sup|F„ im (x) - $(x)| < 



0.7583/(1 -c) + 3.95811 n 1 



kg IF 



Now minimizing °- 7583 /( 1 c)+3. 95811 over c g (0,1), we finally obtain the inequality of the theorem with 
C = 9.257. ■ 
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Chapter 5 

Poisson approximation 



In the first section of this chapter, we leave the coupon collector's problem and concern Poisson approxi- 
mation to the distribution of sums of independent nonnegative integer valued random variables in general. 
We complement the classical Poisson convergence theorem of Gnedenko and Kolmogorov [15], in the setting 
of triangular arrays, with error bounds that are expressed in terms of total variation distance, which was 
defined in Section 2.1. as 

<hv(X,Y) = sup \P(X G A) -P(Y G A)\, (5.1) 

ACZ+ 

for any two random variables X and Y that map into Z + := {0, 1, . . .}. 

For each n, we approximate the distribution of the n-th row sum with a Poisson distribution whose 
mean A„ is defined only in terms of the distributions of the random variables in the n-th row, namely X n 
equals the sum of the probabilities P(X ^ 0), where X runs over the random variables of the n-th row. We 
do not assume the existence of moments, as is the case in analogous results proved by Barbour and Hall [4], 
and our lower bounds are much simpler in form to theirs, being of precisely the same form, up to a constant, 
as our upper bound, provided that the means A n are bounded away from infinity. 

We then continue in the second section of the chapter with an application of these results to the 
coupon collector's problem. We recall from Section 1.2. that Baum and Billingsley proved in [8] (using the 
method of characteristic functions) that if 

71 — TTL i 

m — >• oo and — — > v 2A for some A > constant, as n — > oo, (5-2) 

then W n _ rn — (n — m) converges in distribution to the Poisson law with mean A. We express this problem 
as a special case of the Poisson limit theorem above, and immediately obtain the corresponding Poisson 
approximation results. An even stronger result can be proved in this special case: due to the combinatorial 
structure of the problem, one can determine explicitly the first order term in the error of the approximation, 
and this is what we shall do in Section 5.3. 

Finally, in Section 5.4. we give another Poisson approximation result to the coupon collector's waiting 
time. This time the mean A^ of the approximating Poisson law is chosen to match the mean of the waiting 
time. The result is proven with the help of Stein's method. 

5.1 Poisson approximation in a general Poisson limit theorem 

In [15] (p. 132) Gnedenko and Kolmogorov give necessary and sufficient conditions for sums of independent 
infinitesimal random variables to converge to the Poisson law. In case of nonnegative integer valued random 
variables their limit theorem can be stated as follows. 
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Theorem 5.1.1 (Gnedenko, Kolmogorov) Let {Y n i,Y 7l 2, . . . , Y nrn } n ^ be a triangular array of row-wise 
independent nonnegative integer valued random variables such that 

min P(Y nk = 0) -> 1, n -> oo, (5.3) 

l</c<r n 

> constant, n — > oo (5.4) 

fc=l 

^™ 

^P(F„ fc > 2) 0, n^oo. (5.5) 

fc=i 

XTien 

Y n :=^y„ fc AtV a 
fc=i 

as n — >• oo, where N\ is a Poisson random variable with parameter A. 

We shall refine the obvious approximation of the Y n -s that the limit theorem suggests by approximating 
the distribution of each of the Y n random variables not with the limiting Poisson distribution, but with a 
Poisson distribution that has a suitably chosen parameter that depends on n, namely by the distribution of 
N\ n ~ Po(A„), where 

X n = Y,P(Y nk > 1). 

fc=i 



Theorem 5.1.2 (The upper bound.) For any triangular array \y r <ni 1 , . . . , Y nrn }n€N of row-wise in- 
dependent nonnegative integer valued random variables 

d TV (D(Y n ),Po(\ n )) < [ p ( Y nk > 2) + P{Y nk > l) 2 ] . 

k=l 

Proof. The proof follows the argument in [5] p. 181. For each k = 1,2,..., r n , n <G N, we define the random 
variable 

0, if Y nk = 0; 

1, if^fe>l. 



Ink '■ = 



Thus for each n £ N, I n := X)I-=i Ink is a sum of independent Bernoulli random variables with success 
probabilities q nk := P(Y nk > 1), k = 1, 2, . . . , r n . By Le Cam's inequality [19] 

d TV (V(I n ), Po(A n )) = £ P ( y ™^ ^ X ) 2 - 

fe=l fe=l 

Also, for any two random variables X and Y defined on the same probability space the coupling inequality 
(2.16) says that 

chv(V(X),V(Y)) <P(X ?Y), 

hence we have 



d TV (V(Y n ), V(I n )) < P £ Y nk £ 7 »* = P ( U kUi Y nk ¥= Ink}) 

\ k =l k= \ / 

< g P(Y nk ? I nk ) = £ P(Y nk > 2). 

k=l k=l 

Putting these two bounds together in 

thv(P (Y n ),Po(\ n )) < d TV (V(Y n ),V(In))+d T v(V(I n ),Po(X n )), 
the assertion of the theorem follows. ■ 
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Theorem 5.1.3 (The lower bound.) If {Y n i, Y n 2, ■ ■ . , Y nrn } n ^ is a triangular array of row-wise inde- 
pendent nonnegative integer valued random variables such that 
min 1 <j,< rji P(Y nk = 0) > § for all n £ N, then 

d TV (V(Y n ),Po(X n )) > i ( J\ P(Y nk =0))jr [P(Y nk > 2)+P(Y nk > l) 2 ] . 



\k=l / k=l 



Before turning to the proof of Theorem 5.1.3 we prove a simple result we are going to need later on. 
Proposition 5.1.1 If < yi < x-i < 1 for all i = 1, 2, . . . , n, n S N, then 

/ n \ n n n n 

n yi ) ~ ^ - n Xi n ^ - xx t?; ~ 



/ i—1 2—1 i— 1 i—1 



Proof. Defining t/o : = 1; we can write the difference of the two products in the form of a telescopic sum, 
thus 



7 x < ~ TT Vi = y^bi • • • Vk-iXk ■ ■ ■ x n - yi • • ■ ykXk+i ■ ■ ■ x„] 

»=1 j=l fe=l 

n 

= yiOzfc - ?/fc)(yi • • • Vk-iXk+i • • • x n ). 
fe=i 

Due to our assumption on the y^-s the last expression can be bounded form above and from below by 

n n n 

yi ■■ ■ Vn ^2(xk - yk) < ^2( x k - yk){yi ■ ■ ■ yk~ixk+i ■ ■ ■ x„) < ^2(x k - yk), 

k=l fe=l k=l 

and the assertion follows. ■ 

Proof of Theorem 5.1.3. We introduce the notations P(Y nk = 0) = p nk and P(Y nk = 1) = (1 — p n k)Pnk, 
k = 1, 2, . . . , r n , thus A„ = Ylu—i ~Pnk), n £ N. We are going to prove the theorem by approximating the 
following elementary lower bound for the total variation distance of the distributions considered: 

d TV (v(Y n ),Po(\ n )) > l|p(y n = o) - p(N Xn = 0)1 + i|p(y n = 1) - P(N Xn = i)|, (5.6) 

which can be justified by taking A = {0} and A = {1} in (5.1). 

We start by bounding the difference of the point probabilities at 0. Since 



P(^a„ = 0) = c- A " = fle"^ 



^-(i-P„ fe ) 

fc=i 



P(Yn =0) = Y[pnk, 

fc=l 

and e _ ^ 1_p ™ fc - ) > p nk for all fc = 1, 2, . . . , r n , neN, applying the Proposition above yields 
|P(7V An = 0) - P(Y n = 0)| > ( f[p n k ) £ [e-t 1 "^) 



V&=1 / fc=l 



Since 1 — p n fc < 1, we have 



-(l-P-fc) > 1 - (1 - p„ fc ) + 1 (1 - p„ fe ) 2 - 1 (1 - p nk f > p nk + Pnkf 



40 



for k = 1, 2, . . . , n, n € N, which yields 



|P(^A„ = 0) - P(Y n =0)| > | ( Pnk J E (1 - Pnkf 

\k=l ) fc=l 



(5.7) 



This inequality implies the assertion of Theorem 5.1.3 in the case when 3 J2fc=i (1 — Pnk) 2 
— (1 — Pnk) (1 — Pnk), because we can bound 3/5th of the sum in the display above using this 

assumption. In fact in this case we obtain a better bound than the one we aimed at. Otherwise, if 
2 y^l— i (1 — Pnk) (1 — Pnk) > 3 Sfc=i (1 ^ Pnk) 2 , we need to examine the point probabilities at 1 too to 
improve our current bound. 

We have 

P(N Xn = 1) = A„c- A " = £(1 - p nk ) exp \ - g(l - p nk ) \ , 



k=l 



and since for an arbitrary n £ N Y n = 1 iff for k = 1, 2, . . . , r„ exactly one of the l^fc-s takes on 1 and the 
rest take on 0, 

(1 -PnkjPnk 



p(i r n=i)= E 



^fc=l / fe=l 



Pnfc 



Some elementary algebra gives 



p(tv a „ - i) - P(y„ = i) = J] P „ fc Pnfc ) (i - p nk ) - £ 



(1 - Pnk) ) Pnk 



k=l \k=l 



k=l 



Pnk 



exp < - ^2(1 -Pnk) \ - ]\Pnk _ ^' lfe )' 



k=l 



Pnk 



where in the second term we recognize the point probabilities at 0. Using < 



k=l J k=l 
1 



and the fact 



Pnk mm p nk 

Kk<r n 



that the difference of the probabilities in the formula above is always positive we obtain 

P(iV A „ = 1) - P(Y n = 1) > IT P"* E (! " C 1 ~ = £( J ~ 

\ nun p„fc 

fc=l \fc=l l<fc<r„ fc=i 



From this by inequality (5.7) we obtain 

P(^a„ = 0) - P(F„ = 0) + P(N Xn = 1) - P(F„ = 1) > 



I fe=l 



3 min p nfc 

l<fc<r„ 



E(!-^) 2 



fc=i 



Now ^ > — 1 in the range of n for which the assumption mmi<k<r n p„k > § of the Theorem 

holds, thus 



P(iV A „ = 0) - P(Y n = 0) + P(iV A „ = 1) - P(Y n = 1) > 



1 Pnfc E ^ ~ Pnk > ( X ~ P" fe ) ~ E^ 1 ~ Pnh 



Vfc=l 



fe=l 



and it can be seen that the latter bound is at most 



\ \f[P" k ) \T,( 1 -Pnk)(l-Pnk)+Y / (l-Pnk)' 



\k=l 



\k=l 



k=l 
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for all n such that ^Y^k=i (1 ~ Pnk) (1 — Pnk) > 3J2k~i (1 ~ Pnk) 2 - This together with (5.6) proves the 
Theorem. ■ 

Theorems 5.1.3 and 5.1.2 together state that the order of the error of our Poisson approximation for 
the random variables in Theorem 5.1.1 is 

r n 

£[p(W{o,i}) + p(y, lfe >i) 2 ]. 

fc=i 

Barbour and Hall have proved similar results in [4] using Stein's method: they approximate a sum 2J=i °f 
independent nonnegative integer valued random variables with a Poisson variable that has mean 5ZJ=i P(^j = 
1) or 53" = i E(Yj). (Note that the parameter of our approximating Poisson random variable is between these 
two values.) Their bounds are expressed differently, and involve second moments of the random variables Yj. 
Moreover, their lower bounds would yield no useful information at all in the application to be considered in 
the next section. 

We also obtain the following result. 

Corollary 5.1.1 For the rate of convergence in Theorem 5.1.1 we have the upper bound 



d TV (V(Y n ),Po(X)) < £ [P(Y nk > 2)+P(Y nk > l) 2 ] 



k=l 



£p(F„ fe >l)-A 



k=l 



, n G N. 



Proof. Since 

d TV (V(Y n ),Po(\)) < d TV (V(Y n ),Po(X n ))+d TV (Po(X n ),Po(X)), 

the assertion follows from Theorem 5.1.2 and because for any N Vl ~ Poisson(^i) and N U2 ~ Poisson(^2), 
where < v\ <V2, we have 

d TV (V(N Ul ),V(N U2 )) < min{l,i/ 2 - 1/2 } (u 2 -v x ). 
For reference see for example Remark 1.1.4. in [5]. I 



5.2 Coupon collecting with an approximately Poisson 

distributed waiting time application of the general results 

We begin this section by examining how the coupon collector's problem defined in the introduction fits 
in the framework of the previous section. The equality in distribution in (1.1) can be reformulated for 

W n<m := W n<m - (n-m) as 

n 

i— m+1 

where the X„_i, i = m+1, . . . , n, random variables are independent, and X„ t i + 1 has geometric distributions 
with success probability i/n, i G {m+1, . . . , n}, n G N. The triangular array {X„.„ i+ i, . . . , X„. n }„ e N satisfies 
the conditions of Theorem 5.1.1: the variables of the array are infinitesimal, i.e. they satisfy condition (5.3): 
for any < e < 1 

max P(X n ,i > e) = 

m+l<z<n 

by (5.2); and according to (5.9) and (5.10) in the proposition below, they also satisfy conditions (5.4) and 
(5.5). ' 



1 - min P{X n 

m+l< i<n 



o) 



1 — min — 

m + l<i<n n 



n — m + 1 
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Proposition 5.2.1 If {m = m(n)}„ e N is a sequence of integers that satisfies (5.2), then 

K = A„,i := E ( 1 ) an <^ ( 5 -9) 



n , 

i—m-\-l v 

j'-l 



Vj ■ := X! - A "(^) " ' and A »J^ ' J =2,3,... (5.10) 



i—m+l 



Proof. (5.9) is true, because 

A n = E (l — — ] = n — m— — 
L — ' » n I n 



n(n + 1) m(m + 1) 



(n — m) (n — m — 1) 
2n 



A 



i—m+l x 

by (5.2). By taking the square root of both sides of the equality above it can be deduced that 

n — m — 1 



< V2A„. (5.11) 
y it 

Now we prove the first assertion of (5.10) by induction. For an arbitrary j = 2,3, .. . we bound X n _j as 
follows: 



n — m — 1 +^ f i \ 3 1 n — m — 1 , 
A„ = > 1 - - < V 1 - - = A 



n 

i—m+l i— m+1 



*n,j— 1 

/t \ /t y /i 

i— m+1 

Since for j = 2 this gives A„.2 < by (5.11), we have the first part of (5.10) in this case. If we have 

the same result for some j > 2, then it holds true for j + 1 as well by the argument above, (5.11) and the 
inductional hypothesis. Since A„ — > A by (5.9), the second part of (5.10) follows from the first. ■ 

Thus we see that the limit theorem proved by Baum and Billingsley [8] concerning the coupon collec- 
tor's problem is a special case of the Gnedeno-Kolmogorov theorem. If we apply the results of the previous 
section to W n>m , we obtain the following. 

Corollary 5.2.1 If {m = m(n)} ng N is a sequence of integers that satisfies (5.2), then the error of the 
approximation of the coupon collector's W„ |m waiting time with the Poisson random variable N\ n , that has 

mean A„ = J2i= m +i i 1 ~ > is °f order Y!i= m +i i 1 ~ ■ In f act ' f or al1 n such that min m+i<i<« £ > f , 



l( n - n e 1 1 4) <d^{v{w n , m ),v { N Xn ))<2 e f 1 -^ 

\i=m+l / i— m+1 i— m+1 



Corollary 5.2.2 For the rate of convergence in the Poisson limit theorem concerning the coupon collector's 
problem we have the upper bound 



d T v(V(W n . m ),V{N x )) < 2 



n 

E 

2=771+1 



1 - 



n 

E 

7 = 771+1 



1 - 



- A 



5.3 Coupon collecting with an approximately Poisson 
distributed waiting time — combinatorial approach 

Now we undertake a combinatorial approach to the coupon collector's problem, which will yield us a stronger 
result than the one of Corollary 5.2.1. Namely, we shall derive the first asymptotic correction of the 
P(W„ )m = k), k = 0,1,..., probabilities to the corresponding Poisson point probabilities. We state the 
result in the following theorem. We note that in principal the method presented in the proof can be 
extended to determine higher order terms in the asymptotic expansion. 
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Theorem 5.3.1 If {m = m(n)}„ e N is a sequence of nonnegative integers that satisfies (5.2) and X n and 
X n _2 are defined as in (5.9) and (5.10), then 



P(W„, r 
P(Wn,r 



0) = e" 

1) = e" 
k) = e 



fc! 



-A„ ^n,2 

2 



O 





:=) 










(fc-2)! 


fc! 



k > 2. 



We note that A„ :2 = ^7T~ + °(n)- In deed, 

n , 



i—m+l 



= n — m 

n 



n(n + 1) m(m + 1) 



2 2 
(n — m)(n — m — l)(n — m — j) 

(n — m)(n — m — 1) 



n(n + l)(2n + 1) m(m + l)(2m + 1) 



(2A„) 



3/2 



3^n 



3n 2 



n — m — — — yjn — m)(n — m — 1) 



where we used the fact that A n = — tMSl m 1 ) ; which we calculated in the proof of Proposition 5.2.1, and 
the second term in the formula above is O(^) by (5.2). 

Proof of Theorem 5.3.1. We are going to represent each possible outcome of the collector's sampling 
with a sequence of integers the following way: let us suppose that while sampling (with replacement), the 
collector labels the distinct coupons he draws form 1 to n — m in the order he obtains them in the course of 
time, and after each draw he writes down the label of the coupon just drawn. So he begins the enumeration 
of labels with a 1 after the first draw, and each number that he writes to the end of his list after a draw is 
either the label already on the coupon he just got (if he had drawn the same one before), or it is the label 
he gives the coupon at that moment, which would be the smallest positive integer he has not yet used in the 
process of sampling and labeling. In the first case we call the new member of the sequence "superfluous", 
while in the second case we call it a "first appearance". 

We fix an arbitrary fc € N, and we suppose that n so big that n — m > k holds. Now W n m = k means 
that the collector had fc "superfluous" draws, thus the corresponding representing sequence contains n—m 
"first appearances" and fc "superfluous" members. We categorize all such outcomes according to how the fc 
"superfluous" draws are split into blocks by the n — m "first appearances" in the representing sequences: to 
each vector fc = (k m +i, fc m +2, • ■ • , k n -x), where k% G Z+, i = m+1, . . . , n— 1, and X)"=m+i ki = k, correspond 
the sequences where there are fc„_i "superfluous" members between the 1st and 2nd "first appearances", 
fc„_2 "superfluous" members between the 2nd and 3rd "first appearances", and so on, fc m +i "superfluous" 
members between the (n — m— l)th and (n — m)th "first appearances". (This is the same as saying that 
X n i = ki, for all i = m + 1, . . . , n.) The probability of getting such a sequence is 



n ( n — l\ n 1 n — if n — 2 

- 1 1 

n \ n I n \ n 



k n - 



m + 1\ km+1 m+1 



n I n * 

\i— m+1 / i— m+1 x 



It follows that 



nw n , m = k)=( II; E II 1 



\i—m-\-l / AiGifc i— m+1 



(5.12) 
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where 

( n-1 



k e z\- m ~ l : J2 k * = k 



i—m+l 



Now we are going to examine the sum in (5.12) above, which we denote by S n . m .k = Sk- For k = it 
is an empty sum, and thus it equals 1 by definition. Now let us suppose that k > 2, we are going to return 
to the cases k = and 1 later on. For an arbitrary such k we see that 

Ik = uf =1 /fc.i, where Ik,i = {k<E Ik '■ k has exactly I nonzero components}, I = 1, . . . , k, 

and we correspondingly define Sk t i to be the part of Sk that contains the summands over k £ Ik,i, thus we 
have 

n — l , ■ \ ki k n — l , ■ \ ki k 

n t 1 -;) =ee n t 1 -;) = E 5fc . ; - (5-i3) 

kel k i=m+l V 7 i=l kelk.i i=m+l V 7 i=l 

To determine the limit of Sk we examine the asymptotic behavior of the Sk,i expressions separately. 
We fix an arbitrary I = 1, . . . , k, and with \A\ denoting the cardinality of an arbitrary set A, we now calculate 
\Ik,i\- We can think of the vectors in Ik as the results of distributing k 1-s in n — m — 1 spaces in all possible 
ways: to each of these distributions correspond a vector in Ik whose ith component is the number of 1-s 
put in the ith space, i = m + 1, . . . , n. To produce a vector in Ikj, we first choose I different spaces, and we 
put a 1 in each of them, then we distribute the remaining k — I 1-s in these previously chosen I spaces that 
already have a 1, but this time any such space can be chosen more than once. This gives 



'k,l 



m — 1\ (k — 1 
I )\k-l 



l = l,...,k. 



We obviously bound Sk,i from above if we replace each of the factors in its products by the largest one of 
them, namely by 1 — This together with the just calculated formula gives 

m — 1\ (k — 1\ / m + l\ k , , . , . / n — m — 1 \ k+l ( 1 N 



I I \k — I J \ n J \ \fn ) \ vn 

Hence by (5.11) we have 

Sk ^W^y^ n l^J and 



J2s k ,i <fc!min{l,(2A n ) fc } \-j=j (5.14) 



for any I' £ {1, . . . , fc}. We see from the first inequality that Sk,i goes to for I = 1, . . . , k — 1, but it gives 
a constant upper bound for I = k. We are going to examine the latter case more carefully. Notice that the 
components of a vector in Ik.k are all 0-s and 1-s, thus for any fc € Ik k % ft — ^ — z r — 1- Using this 

' "<m + 1 • "*m + 2 "-Ti — 1 ■ 

and the decomposition of the index set Ik = Li k =1 Ik,i we obtain 



1 fc! 



re— 1 / ■ \ ki 



Sk ' k b. e k m+1 \k m+2 \ . . . k n -!\ . n „ 



fc— 1 n—l / . \ ki 

E E I \U ~7j Z 77 IT ( 1 ~~ 7, 

1 = 1 k£l kl i=m+l v 

k 



The first term of Sk.k is equal to y E™=m+i (l — ~)] by ^ ne polynomial theorem, thus we have 



^ k k—l n—l y . \ k 



^=t-E e fc -ife + , fc n u-y • ^ 

1=1 k&I ki i i=m+l v ' 

It follows that lim„_ J . 00 Sk.k = Tr ( because we have (5.9), and the sum above can be bounded by J~JzL=i 
which goes to by (5.14). Thus putting together our results for the expressions Sk,i in (5.13), we conclude 
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that the part of Sk that counts - in the sense that it asymptotically contributes a positive constant to Sk — , 
is Sk,k, which is the part of the sum in the defining formula of Sk that corresponds to the 0-1 vectors of 
the Jfe index set. 

If we write (5.15) into (5.13), we obtain the following formula for Sk- 



Sk 



A 
fc! 



fc-i 



(5.16) 



i=i 



where 



R 



£ 1 



n 

2— m+l 



k m +l l -k m +2 l - ■ ■ ■ fcn-l! 

Our aim is to determine the first order term of the error when we approximate Sk by Since Rkj < Sk,i 
for each I = 1, . . . , fc— 1, and for the latter expressions we have the bounds of (5.14), we see that Yli=i Rk,l = 
O an d the same, but more detailed argument also gives 

k-2 k-2 

Y,R k ,l < E^< ^ fc!min{l,(2A n ) fe } -. (5.17) 



1=1 



Thus the leading term of the error 



Sl- 



js of order and it comes from the term Rk,k-i- 



Before examining Rk.k-i we introduce some notations for further use. As an analogue of the set Ik,i 
we define Ik-2.1 to be the set of vectors fc £ Z"~ m_1 such that Y^i=m+i = k — 2 and k has exactly I 
nonzero components, I = 1, . . . , k — 2. Also, as an analogue of the expressions Sk,i and Sk we define Sk—2,1 
and Sk-2 by the formulas in (5.13) with k replaced by fc — 2. Finally we introduce 



1 k-2,k-2 



{fc £ l k -2.k-2 ■ kj = 0} , j = m + 1, . . . , n. 



We now return to R^k—i- The corresponding index set Ik.k-i contains vectors that have exactly one 
component equal to 2, fc — 2 components equal to 1, and the rest 0. Thus we have 



R 



k,k-l 



e n 

kel k ,k-l 2=771+1 



We can write Rk,k-i in another form, if we first sum according to the component of the vectors in Ik.k-i 
which equals 2: 



Rk.k-i 



1 

5 £ 

j — rn + 1 



n-l 



e n 



, keli 



i—m-\-l 




1 ™ 

- E 

2 ^ 

j=m+l 



e n 

fee/fc-2,fc-2 i=m+l 



1 

n 



E 



71-1 

n 



1 - - 

n 



We recognize Sk-2,k-2 in the first sum in the brackets, thus we can replace it by the formula in (5.15) with 
fc — 2 in the place of fc. As for the second sum in the brackets, we see that kj = 1, so there is a 1 — £ factor 



4G 



in each of the products, which we can bring before the brackets. These considerations lead to 



2 - k-2 



E M v 



2 ^ V n/ (fe-21! 

j=m+l x 7 v y 

2 n / j ; \ 2 ^ n—1 / j \ 

~ 2 . E ^ ( 1 " n J ^ Wl^TTfc^! . II - n 

j=m+l x ' 1=1 feS/fc-2,; i=m+l x 

-, n / . \ 3 n—X / . \ ki 

-5 £ H £ n H 

• J=m+1 fce-f fc -2, fc - 2 \^_ 2 . fe „ 2 l = m + 1 ^^' 
= : 2 E (l-y (^I-^-^-t (5-18) 

j=m+l v 7 v 7 

Now we bound the last two expressions. First, 

j=m+l V 7 / = 1 V Z 

by (5.10) and the second inequality in (5.14) with k replaced by k — 2. Next, 

n-m-1 A A A 2 2 k - 2 \* 1 



j=m+l v 7 v ; 

by (5.11), (5.10) and the first inequality in (5.14) with k replaced by k — 2 and I = k — 2. 
We conclude that if we write (5.18) into (5.16), we obtain 

X k 1 " / A 2 A fe ~ 2 k ~ 2 

i=m+l v 7 v ' i=l 

where + i? 2 . fc -i + £fr 2 #fc,* = O (£) b Y (5.19), (5.20), (5.17) and the fact that A„ ^ A by (5.9). 

Thus 

a;; i A (. i\ 2 Air 2 , n 



Now we return to (5.12), and approximate the product 0™=m+i n m ^ ^y c _A ™. Using the definition 
of A„ in (5.9) and the expansion formula of the logarithm function the error of the approximation can be 
written in the form 



i— m+1 



n £=h- e i^n-H E 



„ i—m-\-l 



l-i 



j=3 



exp 



J=2 



j=2 



where the expressions A rl .j are defined as in (5.10). Thus we have 



z— m+l 



z— m+1 



■ \ 2 



II ;-" A -5 £ [1~ ) +« 



(5.22) 
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where 



(°° I ( °° 1 °° 1 \ 



and we are going to show that R n = (—). 

We are going to bound the sum in the exponent in R n . Since A„ — > A by (5.9), there exists a threshold 



(5.23) 



number no such that for all n> no we have \/^r < h- This with inequality (5.10) yields 



3-30 



' ^ \ 3-30 



E^ £ ' E ^ ^-m" EU 



,2^ 

n 



for all n> riQ. Let us suppose that n satisfies this condition from now on. 

2 

Now we bound \R n \. First we apply the triangle inequality, then the inequality \c~ x — 1 + x\ < ^- 



valid for all positive real x with x = YJ^ 2 ^A n ,j , and finally use inequality (5.23) with j = 2 and 3. Thus 



we obtain 



\Rn\ <e 



< e 



oo _^ 



J'=3 



exp 



{oo 1 » J 

3=2 J ) 3=2 J 



oo 1 1 / oo 1 

3=3 J \j=2 J 



c- A "4A l (A„ + 1)-. 

n 



Recalling (5.22) we see that we proved 

n 

e -»- n 



i— m+1 



i 



(5.24) 



Finally, recalling (5.12) we have 

p(wu=o)=( n ^)= e - A "-( e - A -- n 



for fc = 0, 



for fc = 1, and 



n / \ -*- -*- n 

i— m+1 / \ z— m+1 , 



P(W n , m = 1) = ( - A„ = e- A "A„ - c- A " - 



n \ n 

^=m+l / \ i— m+1 / 



p(w n , m =k)=[ n - &=e-ss fc - e -*»- n - ^ 



n / \ x x n 

Vz— m+1 / \ i— m+1 , 



for fc > 2. We obtain the first assertion of Theorem 5.3.1 if we write (5.21) and (5.24) into these expressions. 
The second assertion follows from the first and (5.9). ■ 
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5.4 Poisson approximation matching the means 



As mentioned_jit the beginning of this chapter, we shall now approximate the coupon collector's shifted 
waiting time W n ^ m with another Poisson law, namely with the one that has the same mean as W n>m . One 
can easily calculate that in the range of parameters n and m for which the Poisson limit theorem of Section 
1.2. holds true, the error order of this new approximation, given in the theorem below, is 1/n, which is 
clearly better than the error order 1/y/n given by Corollary 5.2.1 or Theorem 5.3.1 for the same case. As we 
shall see, the proof of Theorem 5.4.1 is based on Stein's method, and heavily uses the fact that the means of 
the compared probability measures coincide. We note that the argument presented in the proof of Theorem 
5.1.2 would not work here. 

Theorem 5.4.1 For the coupon collector's shifted waiting time W n ^ m with X' n = Elf„ jm — Y^i=m+i {j — ty> 
we have 



i—m+l 



Proof. Recalling Section 2.3, we apply the Stein-Chen method for Poisson approximation. By (2.15), we 
get the following formula: 

d TV (V(W n . m ),Po(\' n )) = sup \E{X'J A (W n . m + 1) - W n , m f A (W n , m )}\, (5.26) 

ACZ+ 

where f A is the solution to the Stein equation (2.13), and by (2.14), we know that 



sup \f A (k)\ <1aJ^-. (5.27) 

Recalling the distributional equalities in (1.1) and (5.8), we introduce 

C im :=H/„, m -I„ ie{m + l,...,n} 

and 

Wi <m :=W n>m - Xi, i G {m + l,...,n}. 
Taking an arbitrary A C Z+, the two terms on the right hand side of (5.26) can be written in the form 



E 



{\' n f A (W ntm + 1)} = E(W„, m )E {f A (W n , m + 1)} 

n 

i—m-\-l 
n oo 

J2 fcp (^ = fc ) E {Mw n , m + 1)} 



i=m+l k=l 



by (5.8), and 



{W n , m fA(W n ,m)} = E {^iMW n>m )} 

i=m-\-l 
n oo 

= E E E { fc ^(^u + fc )i^ = fc } p (^ = fc ) 

i=m+l k—1 
n oo 



i—m-\-l k—1 
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where we used (5.8) again and the independence of Xi and m . Putting these together, we get 
E{\'J A (W n , m + 1) - W n , m fA(W n , m )} = 

n oo 

E E = fc ) E {MW n ,m + 1) - f A {Wl m + k)) 



i—m-\-l k—1 
n oo 



E E kp ( x * = fc ) E {uw: hm + x t ) - f A {wi m + k)} 

=m+l k=l 

n oo / oo ^ 

E E fcp (^ = fc ) E E + o - wu + fc)i^ = i} p(Xi = i) 

=m+l k=l \l=l j 

n oo oo 

E E E kP ( X > = k + = ^ E + - fA(Wn,m + *) } , 



i=m+l fc=l i = 1 



where at the last step we used the independence again. Thus by (5.27) 

lER/^Wn,™ + 1) - ^,miU(Wn,m)}l < 
n oo oo 



z=m+l fc=l 1=1 



n oo oo 



\ V n / i=m+l fc=l '=i 



Since this inequality holds for each A C Z+, it yields the same upper bound for the supremum of the 
expectations on left hand side taken over the set of functions f A , A C Z+, so by (5.26), we have 



n oo oo 



d T y(V(W n . m ) 1 Po(X' n ))<2[lAJ^-) E EE fcP (^ = fc+1 ) P ( X ' = ; )- ( 5 ' 28 ) 



eX' . 

n / i=m + l k=l ' = 1 



(1 



Keeping in mind the infinite series' sums ^ 1 (1 — a; ) = 1) S^Li J 2 -"' = (l-a) 2 and X^Li ja; 

^Wi+ip f° r an y < x < 1 , we start the calculation of the expression above: 



2j-i 



oo oo oo / . \ k ■ 

££* p (^ = *+ 1 ) p (*« = o = E*( 1 -£) I 

fc=l '=1 fc=l ^ ' 



fe-1 

n I n 



. oo y . \ fc / i \ 2 °° / i N 1 
i 1-r ( i\ 2 1- 



/) 

1 - i 



* (2-*_) 
n — 2 n(n — i) 

~i (2n- if 

{n — z)(4rt 2 — 4m + i 2 — ni) 



i(2n — i) 

(n — i) 2 (4n — i) ( n — A 3 i 2 (4n — i) 



i(2n — i) 2 \ i I J (2n — i) 2 (n — i) 

Now for an arbitrary i £ [m + 1, . . . ,n] the sequence (2ra-i)"(n-t) can ^ e bounded from above by 4. If we 
use this bound for the triple sum in (5.28), we obtain the theorem. ■ 
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Chapter 6 

Compound Poisson approximation 



According to our goals set out in the introduction, in this chapter we approximate the distribution of the 
appropriately centered coupon collector's waiting time with a compound Poisson measure 7r M . a defined at 
the end of Section 1.3. Based on the distributional equality in (5.8), we shall apply general results of 
translated compound Poisson approximation of sums of independent integer valued random variables, which 
has been studied in a series of papers. Using Stein's method, [7] and [2] give bounds for the errors of 
such approximations in total variation distance. Their upper bounds are expressed with the help of the 
first three moments of the summands Xi,X2, ■ ■ . , X n and the critical ingredient c?tv (2? {W n ) , 2? (W n + 1)), 
where W n = YTj=i x o- 

The expression g?tv (2? {Wn) 1 2? (W n + 1)) is usually bounded by the Mineka coupling introduced in 
Section 2.4, which typically yields a bound of order 1/y/n. If the Xj's are roughly similar in magnitude, this 
is comparable with the order 0(1/ \f~VarWn) expected for the error in the central limit theorem. However, 
if the distributions of the Xj become progressively more spread out as j increases, then VarW n may grow 
faster than n, and then 1/y/n is bigger than the ideal order 0(l/\/VarW n ). In fact, this is the situation in 
the case when we chose W n to be the coupon collector's waiting time. 

In the first part of this chapter we shall introduce a new coupling which allows us to improve the 
bounds obtained by the Mineka coupling in such cases. Then, in the second part of the chapter, with the 
help of this new coupling, we shall prove c?tv (2? (W n , m ) ,2? (W n , m + 1)) 

= O (l / y/VarWn.m) t an d therefore that a translated compound Poisson approximation to the collector's 
waiting time with ideal error rate, can be obtained in all ranges of n and m in which a central or 

Poisson limit theorem can be proved. 



6.1 An extension of Mineka's coupling inequality 

We saw in (2.18) that if V r is a sum of iid discrete uniform random variables on the finite interval {1, 2, ... ,21- 
1, 21} for some I > 1 integer, then 

d T v(V(V r ),V(V r + l)) < 



2r 



where \j\pr > l/(ly/r) = 1/VVarK-. However, in the following lemma we show that dTv(T>(V r ),V(V r 
1)) = 1/VVarVr can be established, if we use a new coupling instead of Mineka's coupling. 



Lemma 6.1.1 Let U±, U2, ■ ■ ■ , U r , r > 2, be independent identically distributed random variables with dis- 
crete uniform distribution on {1, 2, ... ,21 — 1, 21} for some integer I > 1. IfV r = Sj=i Uj> then 



d T v(2?(K),2?(K + l)) < 
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S s := (V. + 1) - V' s = 



Proof. We construct a coupling of (V r ,V r + 1). Let U\ be an arbitrary random variable of uniform 
distribution on {1,2,..., 21}. If Ux e {1, 2, . . . , 21 - 1}, then define 

U[ = U 1 + 1 and [/j = Uj, 2 < j < r, 

where U\, . . . , U r are independent; while if U± = 21, then put 

U[ = l and Uj = Uj + lIj, U- = U 3 + 1(1 - Ij), 2 < j < r, 

where Uj has uniform distribution on {1, ...,/}, Ij takes on the values and 1, each with probability 1/2, 
and Uj, Ij, 2 < j < r, are independent, also of U\. 

Introducing V s := Y^j=i Uj an d V' s '■= Sj=i ^j; s e {1, - - - , we see that 

0, if Ui e {1,2,..., 21- 1} 

2' + E-= 2 (^-^). if ^=2/, 

where TT _ TT i = / ^ with probability 1/2, 

J J \ -I, with probability 1/2. 

Thus if U\ = 21, {S s ) r s=1 can be regarded as a symmetric random walk that starts from 21 in time step one, 
and then at each subsequent time step increases or decreases by I. Define T to be the first time the random 
walk hits 0, that is 

T := inf {s > 2 : S s = 0} = inf j s > 2 : Y^Po - U 'j) = ~ 21 

By the reflection principle and symmetry, 

P(T > r\Ui =21) = 1- P(T < r|Z7i = 21) 

= 1 P (5V = Q\Ui = 21) - 2P (S r < 0|[/i = 21) 

= 1 P (S r = 0\U X = 2l)-P (S r < 0\Ui = 2l)-P (S r > <U\U X = 21) 



= ^V(S r = kl\Ui = 21) < 2maxP(SV = kl\Ui = 21), 

k=l 

and by Lemma 4.7 of Barbour and Xia [7], we have 

maxP (S r = kip! = 21) < —= ■ 1 

thus 



Now for j, s G {1, . . . , r} put 



P(T > r\Ui = 21) < -H=. (6.1) 



Of course (V^)^ =1 , (V!.) r s=1 and (V") r s= i all have the same distribution, thus (V" ,V r + 1) is a coupling of 
+ 1), therefore 

<*rv(V r , 7 r + 1) < P(V r + 1 ^ V") = P(T > r) 
by the coupling inequality. Since 

P(T > r) = P(J7i = 2Z)P(T > r|Z7i = 21) < — -= 

lyjr 

by (6.1), the proof is complete. ■ 

Now we show how the result of Lemma 6.1.1 concerning sums of iid uniform random variables can be 
used to obtain similar results for sums of arbitrary integer valued random variables. The idea is to embed 
the uniform random variables in the ones we want to prove the result for. 
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Proposition 6.1.1 If X\, X2, ■ ■ ■ , X n , n > 2, are independent integer valued random variables and W = 
YJj=i x n, then 

d TV (V(W),V(W + 1)) < 7 ^t r + 

ly'nlp nip 

where I G {2,4,6,...} and p < mm{P(Xj = k) : k = l,...,l,j = 1,...,ji} are arbitrary and d„ = 
d T y(V(X n ),V(X n + l)). 

Proof. We write each of the variables Xi, . . . , X n in the form 

X, I J, ■ (1 /,;//,. j = l,...,n, (6.2) 

where Ij, Uj and Rj, j = 1, . . . , n, are all independent random variables defined on a common probability 
space, and for each j = 1, . . . , n: Uj has discrete uniform distribution on {1, 2, . . . , 1} for some even integer /; 
Ij is a Bernoulli random variable with parameter Ip, where p < min{P(Xj = k) : k = 1, . . . , l,j = 1, . . . , n} 
is fixed; and 

f p (^=*>-r , Kk<l, 
P(Rj =k) = \ p^ 1 ^) 7 T k G Z. 

( \_ J ip S otherwise, 

Since ©(X^- = 1) = £>(Uj) and 2?(X,-|/j = 0) = V(Rj), for any £i,...,£ n _i G {0,1} and 
pi, . . . , G Z we have 

P *i I 7 * 'W''.- /',-./' 1 '' i j D(V r + P), 

where r = &j > = S^=i ^> wnere the C/j are independent copies of U\, and are independent of 

everything else, and p = Y^j=i (1 — $j)pj- 
Now we apply the inequality 

d T v(£>(£i),£>(Z 2 )) < E^v^^lZs),!)^!^))} (6.3) 

true for any random elements Z 1; Z 2 and Z 3 defined on the same probability space. We obtain 

d T v{V{W),V{W + 1)) < 

< E^tv^ (f^ X i\IhRh3 =l,...,n-^,V\S2Xj+ 1 /,./,',../ = 1, . . . ,n - 1 

= E{drv(X>(V T + X n + R\T, R), V(V T + X n + R+ 1\T, Rj)}, 
where T = YTjZi Ij an d R = 2^=i (1 — are independent of (t/j, j > 1) and of X n . Hence 

d T v(V(W),V(W + l)) < E{dw(V(Vr+X n \T),'D(VT + X n + l\T))} ) (6.4) 

since total variation distance is invariant under translation. 

Now, since T, X n and (Uj, j > 1) are independent, we have 

d TY (V(V T + X n \T = t),V(V T +X n + l\T = t)) 

< mm{d TV (V(V t ),V(V t + l)),d TV (D(X n ),V(X n + 1))}, 

and Lemma 6.1.1 provides the bound 

d T y(V(V t ),V(Vt + l))<f(t):=l ^ if 1 = 0. 



53 



Writing d n = dTv{T>(X n ),TJ(X n + 1)) we thus obtain from (6.4) that 



d T y(V(W), V(W + 1)) < E{d TV (V(V T + X n \T),V(V T + X n + l\T))} 
<E{min [f(T);d n ]} 



< E 1 2 



< 



[iVT 

2v/2 



ET] „ / ET 

T > > P I T > 

-2| V " 2 



cLP T < 



ET 



EX* 

<i n P ( T < — 
WET \ 2 



Since T has distribution Bin(n — 1, Ip), ET = (n — l)lp > inlp, and by Chebishev's inequality 



thus 



ET\ „ /, , ET\ 4VarT 4 8 



d T y(V(W),V(W + 1)) < + 



(6.5) 



Remark 6.1.1 Since total variation distance is invariant under translation, there is no loss of generality 
in supposing that the l-intervals begin at 1. 



Remark 6.1.2 The choice of (p,l) depends very much on the problem. 



The constants in the upper bound of Proposition 6.1.1 can be improved by refining the method 
proposed in the proof. One could embed not one, but many uniform random variables in the Xj-s by splitting 
the whole line into the /-blocks ({(m — 1)1, . . . , mZ}) m6 z and defining a uniform variable corresponding to 
each block. Thus one could use potential overlaps from the whole distribution and not just the interval 
{1, . . . , I}, when bounding d TV {V(W),V(W + 1)). 

More precisely, each Xj, j = 1, . . . ,n, can be given in the form 

oo 

X j =I jQ R j + Y l I j i(.U ji + (i-i)l), 

i=l 

where all random variables in the decompositions are defined on a common probability space, and for each 
j = 1, . . . , n the following hold true: Uji has discrete uniform distribution on {1, . . . ,1}, i = 1,2,..., for 
some fixed even integer I; Ijo ~ Bernoulli (1 — X^i=i (Pi)s ^ ~ Bernouili(Zpj), where pi < min{P(Xj = fc) : 
k = (i — 1)1, . . . ,il,j = 1, . . . , n} is fixed, i = 1,2,..., and these Bernoulli variables depend on each other 
in a way that for each outcome exactly one of them is 1 and the rest are 0; all the other variables in the 
decompositions are independent of each other and of the Iij-s; and Rj is defined to make the distribution 
of the decomposition equal the distribution of Xj. 

Then, to bound dT\{V(W), T> (W + 1)) we would use (6.3), conditioning on all the Iji-s and Rj-s, 
which would give us (6.4) with T = Y^j=i YnLi hi- 111 case = (n — ^YlTLiPi ancl VarT = 
(n — 1)1 (X^i-Pi) (1 — 'Si=i-Pi)) hence we would obtain (6.5) with p replaced by Y^LiPi- 



6.2 Compound Poisson approximation in the range of the central 
and Poisson limit theorems 

We return to the coupon collector's problem. Taking advantage of the decomposition in (5.8), we apply a 
theorem of Barbour and Xia [7] on translated compound Poisson approximation in total variation distance 
to the distributions of sums of independent integer valued random variables. One of the elements in their 
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approximation error is (almost) dxv(f (Wn,m), f(W n , m + 1)), to bound which we invoke our proposition 
of the previous section. Recalling that in Section 1.3, for ft, a > 0, we defined the compound Poisson 
distribution n^a to be the distribution of Z\ + 2Z2, where Z\ ~ Po(/i) and Z^ ~ Po(a/2) are independent, 
we have the following result: 

Theorem 6.2.1 For any fixed n > 2 and 2 < to < n — 4, £/ 

M = ^« - 2(cr^ - fin), 

a = (a^ — fi n ) and (6-6) 

c = [°n - MnJ > 

where (x) and \_x\ denote the fractional and integer part of x respectively, then there exists a positive constant 
C such that 

a /V> cw j_ ^ \ ^ C ( L g " - - (n ~ m)\ (n-m) 2 \ 

d T v V (VK„, m + c) , Tr^.a < — -t i + . (6.7) 



Remark 6.2.1 We recall one of the Baum-Billingsley theorems from Section 1.2: if m — m € {0, 1, . . . , n- 
1} is an integer that depends on n in such a way that 

n — to 

m — > 00 and -= 5- 00 as n — > 00, 



then W n ,m '■= (W n ,m — EW n , m )/ y/"VarW nim has asymptotically standard normal distribution. This limit 
theorem was refined in Chapter 4 by showing that 

n 1 

d K to(W„, TO ),N(0,l)) < C , 

m a n 

where C = 9.257. One can deduce that the same or better order of approximation is obtained in the discrete 
approximation given in our theorem, than with normal approximation, and now with the error measured with 
respect to the much stronger total variation distance. 



Remark 6.2.2 Note that, with these parameters, 7T M , a has mean fi + a = a 2 n — (a„ — fi n ) = fi n + c and 
variance ft + 2a = <7„. 



Remark 6.2.3 We can express the bound of Theorem 6.2.1 more intuitively with the help of the asymptotic 
formulae given by Baum and Billingsley in [8] for the variance of the waiting time: if n — > 00 , then 

a -> 0, =>. al^zL 

n 3 n m 

f^c,ce (0,1) o-l~ in , 

n 1 n 2 n ' 

where 7 = (1— c+c log c) / c. In i/ie latter case we distinguish two subcases: the case when lim inf ra ,m->oo Wn — fJ-n 
1 and when limsup n m-yoo IctjJ — fi„ — (n — to) J < 1. We s/ia/^ refer to the four categories above as "small", 
"medium", "large" and "very large" to. By these formulae, (6.7) is equivalent to 



TV P W„, m + C ,7Tu„ = < 




m i/ie "small " to case; 

m i/ie "medium" to case; . . 

(6.8) 

m i/ie "large " to case; 

m £/ie "-yery /arge " to case. 
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Proof of Theorem 6.2.1. We apply Theorem 4.3 in [7], which states that if Zj, j = l,...,r, are 
independent integer valued random variables with E|X,| 3 < oo, W = Xw=i Zj> anc ^ we define 



e c (W) 



1, if W + c > almost surely; 
0, otherwise; 



^ := E\Zj(Zj - l)(Zj - 2)| + \EZj\E\Zj(Zj - 1)| + 2E\Zj\\VarZj - EZj\, 
d+ := max \chv(V(Wi),V(Wi + 1))}, where Wi :=W-Zi, 

l<i<r I J 

then with n = VavW - 2{VarW - EW), a = (VarVK - EW) and c = [VarW - EW\ , 

2e c (W) + 2 (| [VarW - EW\ | + ££ =1 ^+ 
d T y(v(W + c)^ a )< V ^-^ 1—. (6.9) 

We apply this theorem with Zj = Xj — 1, j G {m+1, . . . , n}, for the X,- given in (5.8), in order to approximate 

the coupon collector's shifted waiting time W n . m := Y^j= m +ii-^-j — 1]' ana - then show that the upper bound 
in (6.9) is not greater than the right hand side of (6.7). Then, since the two measures compared in (6.9) are 
the same for W = W n , m an d W = W n , m = W n , m + n — m, the theorem for W n ^ m follows immediately. 

To do so, for given n > 2, 2 < m < n — 4 and j G {m + 1, . . . , n}, we bound ipj and d+ as defined 

above. 

For X, a random variable that has geometric distribution with parameter p, we have 

EX = i EX 2 = ^, EX 3 = p2 - 6 f + 6 , andVarX = l^. 

p p^ p 01 p^ 

Thus for Z = X — 1, one can easily calculate 

^ := E\Z(Z - l)(Z - 2)| + \EZ\E\Z{Z - 1)| + 2E|Z||VarZ - EZ\ 

= E{X 3 - 6X 2 + 11X - 6} + E{X - 1}E{X 2 - 3X + 2} + 2E{X - l}|VarX — EX + 1| 
10(1 -p) 3 

pes 



so = 10 f j — l^j . If we add the ipj together, we obtain 



± iPj= T 10(1 - 3 » )3 <10 ( " )2 V ^^ = 10^^ (6.10) 
^ J ^ (L\ nm ^ j 2 nm 

j=m+l j =m +l \ n ) j =m +l 

Next, we notice that e c (W n , m ) = almost surely. Indeed, < cr 2 — EW ntm < a 2 , because for each Xj 
geometric random variable of parameter j/n we have VarX, — E(Xj — 1) = f 1 ~/ T {" ) < (Vr/)" = VarXj. 



J'/" - (i/n) 2 _ »°^\r 

Now combining the bound in (6.10) with inequality (6.9) applied to the W n _ rn waiting time, we obtain 



(V(w n , m + c),^ & ) < 20 ( H-^-Jn-m)\ + (n-^f j ^ ^ 



where c, p and a are defined by the formulae in (6.6) with W nm replaced with W n 
Before turning to the approximation of d + , we bound cr 2 . We see that 



9 v -> n—j n \ . . n(n — m)(n — m — 1) 

j'=rrH 



" j 2 "(m + 1) 2 ^ v J ' 2(m+l) 2 

i+l ■' v ' j=m+l 

also, 

9 n—j , , /" n 1 , n(n — m — 1) 



j— m+1 



— y^- < n(n — m — 1) / — c?^c < 
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thus 

2 f 7i — m 1 , 

ct„ < n(n - m - 1 mm <^ — — ^ , — f . (6.12) 

^ 2(m + 1) m 

Now for d + , by an inequality of Mattner and Roos [21] we have 

d T v(V(Wi),V(Wi + l))<^^ [i-^vPPO^PO + i))] 

and since d,Ty(T>(Xj),T>(Xj + 1)) is equal to 



-DPft = t )-P(X i = i -l)| = -^(l)g(l-i) J_i, (6.13) 



we obtain 



n / m+i<i<n \ 71/ I V 7T ^(n - to - l)(n - m - 2) 



It follows from this and (6.12) that for any X > 

2if 1 n 

d + < — — , if— <TO<n-4. (6.14) 

Putting this bound into (6.11) gives a result which, when compared to (6.7), has an extra factor K > n/m. 
Thus it is of inferior order if to <C n. To prove the theorem for such values of to, we need to use our 
proposition to bound d+. 

Let us assume that 2 < m < \. If we apply the Proposition to the random variables {Xj,j = 
to + 1, . . . , 2m, j 7^ i}, i £ {to + 1, . . . , 2m} fixed, with 

J [-J, ifL-Jiseven, / 2to 

I lIj-i, if Llj is odd, and p - 1, 1 — 

we obtain 

*v(l>( £ x\v{ t Xj + lU<- 7E ' + , (6.15) 



ly/(m-l)lp (m-l)lp 



where 

d T v {©(Xa™), 2?(X 2m + 1)} = if * ^ 2m, 

dTVp^^O^^n-i + 1)} = if i = 2to 

by (6.13). For any i e {to + 1, . . . , 2to} we have 



since [x\ — 1 > it x > 2, and 



2to n 

a < and I > - — , 

n 2to 



2to\ m m e 

lp>7T 1 — > -5-) 

2m \ 7i In I 



because (1 — x)* decreases as x increases in (0,1), and its limit at is e 2 . Now putting the bounds above 
together in (6.15) yields 



j=m+l,j^i I \j=m+l.j^i 



< 8 ^e + 32c 2 , - , < 16c^ + 64e 2 i < (16c + 64c 2 ) 

V7n — In (m — 1)71 77 71 71 
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where the last two inequalities hold for m > 2. By (6.12). < J_ 5 thus we have 

1 n 
d+ < (16e + 64e 2 ) — , if2<m<-. 

c n 2 

This and (6.14) with K = 2 substituted into (6.11) yield the theorem. ■ 
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Chapter 7 

Poisson-Charlier expansions 



In the final chapter of the thesis we approximate the coupon collector's shifted waiting time W n , m = 
W n ^ m — (n — m) with Poisson-Charlier signed measures in total variation distance. To do so, we shall 
apply a characteristic function technique proposed in [6]. Throughout the chapter C and Cr denote positive 
constants, not necessarily the same ones at different occurrences, the first one is always a universal constant, 
while the latter one depends on R. 
Let fi = V(W n , m + c), where 



c = [VarW n . m - EW„, m J = [o* - [n n - (n - m)]J 



E 

,k—m-\-l 



- k 



Recalling the distributional equality (5.8), we see that since the characteristic function of the geometric 
distribution with success probability p <S (0, 1) is e lt /(l — -^^(e 1 * — 1)), the Fourier-Stieltjes transform of fx 
is 



4>{t) : = 



J -° k=m+l \ L k V e L ) J 



= exp|- log^l-^^^-l)) jexp|i* 

Introducing the new variable w = w t ~ c 1 * — 1 and the sequences 

'n- k\ j 



E 

,k—m-\-l 



n — k 



k—m-\-l 



, 3 = 1,2,... 



(7.1) 



(7.2) 



we write <p(t) in the form 
4>(t) = exp < 



£ log(l-^(e«-i; 



k— m+l 



E 

,k— m+l 



n — k 



exp < - ^2 log ( 1 



n — k 



w + [a n ,2\ log(l + w) 



K k=m+l 

Assuming \t\ < m/n, for any k = m + 1, . . . ,n — 1 we have 



e 11 - 1 < 



n — m — 1 . 



II < 



n — m — 1 



1*1 < 1, 



k k 1 m + l 1 m + l 

which together with \w\ = |e l4 — 1| < \t\ < 1 allows us to expand the logarithmic expressions in <f>(t), therefore 



,r=l 



4>(t) = exp \ Y, [ a ^,r + (-l) r+1 L<V 2 J 
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w 
r 



t < 



We note that a n ^i = /x n — (n — m) and a n ,2 = ^ ~~ [f^n ~ ( n ~ m )L hence the line above can be rewritten as 
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^)=x(i)exp{/ l H} ) \t\<- : (7.3) 

where 

x(i)=ex P K(e it -l)} (7.4) 
is the characteristic function of the Poisson distribution with parameter tr^, and 

2 oo r 

h(w) = -(a Hi 2 ~ la n ,2^jw + (a n , 2 ~ [a n .2\)-ir + ^ ( a "< r + ( _1 ) r+1 L a n,2jJ— ■ (7.5) 

r=3 T 

Now we fix an integer R > 3, and modify the function exp{/i(u>)} in (7.3) in two steps, each time 
replacing a certain expression of the previous function with the first R terms of its series expansion around 
0: 

exp{/i(w)} sa exp{h R (w)} ps H r (w), (7.6) 



where 



2 T 

h R (w) = - (a na - [a na \^jw + {a n ^ - |_ a n,2j)^- + ^ (a n , r + (-l) r+1 |_a n ,2j ) — (7.7) 



r=3 

and 

(7-8) 

We approximate the distribution [i = T>(W n>m + c) with i/jj, which we define to be the finite signed 
measure on the nonnegative integers whose characteristic function is 

i>(t) = X (t)H R (^ - 1), tel. (7.9) 

Since H R (e l1 — 1) is a polynomial of e 1 * — 1 of the form Y^=o®n)m(e Lt ~ ^Y, where the a,n]m coefficients 

all depend on n and to, it follows that v R = u R (a^, a^L, • ■ • ,a\^m) is a Poisson-Charlier signed measure, 
which, according to (1.4), is defined by 

MA = Po(^){.?} (l + J^-l) r+1 ai%C r (j, al) j , j G N, (7.10) 

where C r (j, <r^) denotes the r-th Charlier polynomial given in (1.5). 



Theorem 7.0.1 We assume a n 2 > 1. For an arbitrary integer R > 3 £/iere exist threshold numbers m R 
and n R depending on R such that if m > m R and n> n R , then 



and 



sup \n{k} - v R {k}\ < C R — = , ifm < - - 1 



1 \ R 

1 \ . n 



2 



R-2 



II 



sup - u R {k}\ < C R - -jj-r, ifm>- 



Before embarking on the proof of the theorem, we prove a sequence of propositions, which we shall 
need later. 
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Proposition 7.0.1 Assume 1 < m < n — 1 and n — rn > 2. 

ft 1 Tl 2 72 2 

Ifm<--1, then — — < er 2 < — . (7.11) 

3 ~ 2 20 m ~ " ~ m y ' 

n 1 (n — m) 2 (n — m) 2 

Ifm>-, then —± '- < a 2 n < 2^ '-. 7.12 

2 24 n n 

Proof. First we prove the upper bounds. Since the terms in the sum a 2 = Ylk=m+i decrease as k 

increases, we have the bound 



n(n — x) 2 f 1 f 1 / n n 
a n < I j dx = n / — j ax — n I —ax = n I 1 — log — 

*^ t/ TD **' t/ TT) 7*1 Tib 



2 9 

Thus we see that — is always an upper bound for <r^. However, if — < 2, one can apply the inequality 
log x > x — 1 — — „ J with x = — to obtain 

1 f n — m\ 2 (n — m) 2 



where at the last inequality we used the assumption m > § 



The proof of the lower bounds is similar. Again, we use the fact that the terms in the sum a 2 

n(n—i 
+ 1 W 



12k= m +i "^2 fc " > decrease as k increases to obtain 



2 f n n(n - x) , 2 r 1 , f 1, / n , n 
cr„ > / ^ °-x = n / — j dx — n —ax = n I 1 — log 

Jm+l x Jm+l x Jm+l X \7Jl + 1 171+1 

Now if > 2, then by the inequality 1 + log a; < 1+1 ° s2 x, x>2, with x = 

, / n n \ ( l + log2\ n 2 In 2 1 n 2 

al > n 1 - log > 1 > > ■ 

" ~ \m +1 m+ll ~ \ 2 J m + 1 ~ 10 m + 1 ~ 20 m 



where we also used m + 1 < 2m. If ^ ^ < 2, we can apply the inequality logo; < (x — 1) 
< x < 2, and ^ — 1 < m < n — 1, which yield 



n lib , \ lib 

< > n — - 1 - log — > n — - 1 



'5 1 n 



171 + 1 771 + 1 / \ 771 +1 / \ 6 3 771 + 1 

^ 1 / 77 — to — 1 \ 2 ^ 1 (77 — m — l) 2 1 (n — m) 2 (n — m — 1 \ 2 ^ 1 (n — rnf 



6 \ m + 1 J 6 n 6 77 \ n — m J 24 77 

where at the last inequality we used the fact that the function (x—l)/x is increasing in x with x = n—m > 2. 
■ 

Proposition 7.0.2 We fix an arbitrary integer R > 3, and define 



t Q := —JirRlogy/m. (7.13) 



There exists a threshold number trr depending on R such that if 777 > wir, then 



1 rn to 2 / 3 77 , 

*o<t-A , ifm<--\. 7.14 

4 77 77 2 



There exists a threshold number nn depending on R such that if n > 77^ and 777 is such that 



n — m 8 

i/ie77 



(7.15) 



1 771 77 2 / 3 77 

to< 7 ~/\ , ifm>-. 7.16 

4 77 77 — rn 2 
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Proof. First, if m < § — 1, then by (7.11) in Proposition 7.0.1, 

t0 < V20^ ^ ml °g^ , (7.17) 

n 

Thus we see that (7.14) holds true if m is greater than some threshold number depending on R. 
Next, if to > ^, then by (7.12) of Proposition 7.0.1, we have 

t < ^/24 ^ nl0g ^^ (7.18) 
n — to 

where the bounding sequence is less than both n 2 / 3 /{n — m) for all large enough n, depending on i?, and 
since m > assumption (7.15) implies to < \ — ■ Therefore we also have (7.16).B 

Proposition 7.0.3 For a ?lj 2 defined in (7.2) we have 

(n — to — l) 3 (n — to) 3 

* -^-^ < a n ,2 < ^- (7.19) 

For a n j , with j = 2, 3, . . ., also defined in (7.2) we have 



n J „ n , • in — toV +1 



a n i < 2 J — if m < — — 1, and a n 7 < 2 J ; , if to > — — 1. (7.20) 

J ~ to-?" 1 ' J ~ 2 ' J ~ n> 2 y J 



Proof. Since a n ,2 = Y^k=m+i ]J> > ^ ne nrs ^ assertion follows from 

(n — to) (n — to — 7;) (n — to — 1) ^ (n—k) 2 , . (n — to — 1) 

V i < a„, 2 < (n - m) v 



3n 2 ■ "* n 2 ' (to + 1) ! 

k=m+l v ; 



For any j = 2,3, 



^ Wy j - 1 \rnJ- 1 / ~ to^ 



/={) 



The approximation applied at the second inequality in the display above is quite gross in the case when to 
is close to n. In this case, with the help of Proposition 7.0.1, one is able to prove the following better upper 
bound for a n j: if to > j — 1, then 



y / " - m < (» - ™ - i) J 1 g 2 < 2 ( n - m ) J+1 < p., (" - m V +1 

I fe / - (m + l) i_2 n CT ™ ~ (to + l)J- 2 n 2 ~ vP 



k— m+1 



Proof of Theorem 7.0.1. First of all we note that by (7.19), the assumption a„2 > 1 implies (7.15) for all 
n that is greater than some threshold number depending on R. Let tur and ur be positive integers at least 
as big as the threshold numbers given by Proposition 7.0.2, and such that (7.15) holds true for all to > tur 
and n > ur. We fix an integer R > 3, as well as integers to > tur and n > ur. 

We shall apply Theorem 2.2.1 with the measures /i and vr and the constant to given above. Recalling 
the decompositions of the characteristic functions corresponding to fj, and vr in (7.3) and (7.9), we now give 
upper bounds for the differences | exp{/i(e'* — 1)} — if^e 1 * — 1)|, \t\ < to, and \4>{t) — ip{t)\, to < \t\ < tt, that 
have the form required by the theorem. 

We begin by bounding | exp{/i(e'* — 1)} — HR{e lt — 1)| when \t\ < to- For an arbitrary such t, 

I cxp{/ l (e it - 1)} - H R (^ - 1)| < A x + A 2 , (7.21) 



02 



where the As are the errors resulting from the approximations in (7.6). 
Starting with Ai, the inequality 



l+ c l 22 l) \ Zl -z 2 \, Zl ,z 2 eC, 



|e Zl -e Z2 | < - (el Zl 



Ai = \exp{h(w)} - exp{h R (w)}\ < l(e Ww) \ + e ^ R( - w A \h(w) - h R (w)\ 



yields 



From the definitions of h(w) in (7.5) and that of h R {w) in (7.7), 



\h R (w)\ V \h(w)\ < |w| + ^L+2^(a„ ir Va„ >2 ) 



r=3 



M 
r 



(7.22) 



If m < £ — 1, then by (7.20) in Proposition 7.0.3, (a„. r V a n ,2) < 2 r „"_i , r = 2, 3, . . ., which implies 



\h R (w)\v\h(w)\ < \ w \ + ^± + w ^^± 



\ m 



-3 9, 



32 n 3 



< -\w\ + -5 o\ w 

3 m z 



(7.23) 



At the last inequality we used \w\ = |c'* — 1| < |t| < t n < i— guaranteed by (7.14). If however to > ^, 
then "^"Y 1 < 1, thus each term in the defining sum of a„. r decreases as r increases, which means that 

o-n.2 > On, 3 > ■ ■ •■ Therefore by (7.20), for any r = 2, 3, . . ., (o n/ V a n ^) < a n .2 < 4 ^"~"^ , and hence 



IM«0| v \h(w)\ < H + hr + 8 



\w\ 2 (n — m) 3 \w\ 3 



El 

r=3 



™r~ 3 < -M 



32 (n - to) 3 



(7.24) 



At the last inequality we used \w\ = |e'' — 1| < \t\ < to < |, true because of (7.16). Note that by (7.23) and 
(7.24), (7.14) and (7.16) also imply 

\h R {w)\V\h(w)\ < C (7.25) 

in both cases (that is for all to). 

If we write (7.25) back into (7.22), we obtain 



Ai < C\h(w) - h R (w)\ =C 



E 

r=R+l 



l) r+ >n, 2 J — 



°° / \ \w\ r 

<C ^2 2[an,r V a n ,2J ■ 



r=R+l 



We proceed by bounding the sum in the last expression. Once again we distinguish two cases, according to 
the values of to. If to < — 1, then we use (a Iljr V a„ j2 ) < 2 r m "_ 1 , r = 2,3,..., thus 



r>R+2 R+l 

\h(w)-h R (w)\ < 



R+l m 1 



- 1 E 2 -i 

' — ' V TO 

r=R+l 



r-R-1 



While if to > 2., then for any r = 2, 3, . . ., (a n , r V a„ 2) < fln,2 < 4 — ™' 1 as noticed before, thus 



\h(w)-h R (w)\ < 



(n-mf R+1 



R+l n 2 



E M 

r=R+l 



r-R-1 



The last two sums can be bounded with the help of \w\ < t and (7.14) or (7.16) as seen before, therefore 
we conclude 



Ai< C R *&\w\™ m<f-l; 
1 C R ^pL\w\ R +\ to>|. 



(7.26) 
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We now deal with A2. Recalling the definitions in (7.7) and (7.8), by the series expansion of the 
exponential function, we have 



A 2 = \exp{h R (w)}-H a (w)\ 



00 hi 



E 

l=R+l 



h l R {w) 



n 



< 



\h R {w)\ 



R+l 



(R 



1 y 

1)! ^ 



\h R (w)\ l - R -i \h R (w)\ 



R+l 



l=R+l 



[l-R-l)\ {R + l) 



■cxp{ \h R (w)\}. 



(7.27) 



If m < § - 1, we use (7.25) and (7.23), and then the inequality (a + b) k < 2 k (a k + b k ), a,b e R + , 
k G N, which yield 



A 2 < 



(R + iy. 



9. 32 n A 
oM + ^r — M' 

8 3 777 z 



fl+1 



< C R \w\ R+1 + C R ^-\w\ 



,R+1 



R+l 



III 1 



j2(H+l) 
mR+2 



\2(R+1) 



where 



2(fl+l) 



= c> 



TYl 



II 



by (7.17), and we see that there exists a constant depending on R, which bounds the last expression from 
above for all values of m. 

If to > t| , we use (7.25) and (7.24) to continue the approximation of A2 in (7.27). Also applying the 
inequality (a + b) k < 2 k (a k + b k ), a,b e R+, k e N, we obtain 



A 9 < 



(fl+1)! 



32 (77 — m) 



< C R \w\ R+1 + C R 



(n-m) 3 l lR+1 



3 -ifl+i 

3 



w 



(n-m) 3R 2(R+1) 

„2R l w l 



where 



< (77 - m) 3ii / ynlog(V77) 



2(fl+l) 



^ (n nfl m r 2 (^(^)) 



(-R+i) 



(log(v^)) 



(fl+i) 



by (7.18), and again we see that there exists a constant depending on i?, which bounds the last expression 
from above for all of 77. We also note that in the latest bound for A2, the second term is the bigger one due 
to our assumption a„2 > 1 ari d (7.19). 
These considerations lead to 



A, < 



C R ^-\w\ R +\ m<f-l 
C R ^^\w\ R +\ m >§, 



which, up to a constant factor, is exactly the same upper bound we obtained for Ai in (7.26). Hence by 
(7.21) and \w\ < \t\, 



|exp{/7(e it -l)}-i? fl (e i *-l)| < 



C R ^r\tr\ m<|-l; 
C R ^\t\ R +\ 777 >S, m ~ t0 



(7.28) 
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Next, from (7.4), by the application of the inequality 1 — cost > -^jt 2 , < t < 7T, we see that 

\x(t)\ =cxp cost)} <cxpj-^fi 2 J, 0<i<7r. (7.29) 

The next step in our proof is to bound IVKOI- We recall the decomposition tp(t) = x(t)_ffR(c l4 — 1) in 
(7.9), and start by examining \Hr(w)\ (where w — e 1 * — 1) defined in (7.8) and (7.7). If we look at (7.7), we 
see that with the application of the triangle inequality and \w\ < \t\, \h R (w)\ can be bounded from above 
by a polynomial of \t\ of degree R. The coefficients in this polynomial are less than 1 for \t\° and \t\, and 
their order is given by (7.20) for \t\ r , r = 3, . . . , R. If m < ^ — 1, these orders are n r /m r ~ 1 , r = 3, . . . ,R, 
respectively, and hence 

\hnH\<C + Cn ^|t|3j [0i9](t) + _^_|t|Hj (5|ir] ( t )^ , 

where for any icffi. I a if) is 1 if |t| £ A and otherwise. If m > § the coefficient orders in the polynomial 
are (n — m) 3 /n 2 for all r = 3, . . . ,R, so 

\h R (w)\ <C + C R (^Z^\t\% tl] (t) + ili-^1%^)) . 
By (7.8), these bounds imply 

(3 3R \ 

^\t\ 3 i [o ^ (t) + ^\t\ 3R i^^ (t) + -^itfi^^j , 

if m < § — 1 , and 



+ ^~ 2 11 (i.^vi)W + ~^55 1*1 7 [^vi, OT ]W 



if m > §. 



we get 



We introduce a new variable x n t '■= cr^t 2 . With this, for m < § — 1, by -^^t 2 < a 2 t 2 from (7.11), 

/ 3 3R b£. \ 

l^flHI < Cfl + Cfl hc» f Ir , /s j (t) + X„ 2 t |? ] (*) + I«t J 

=:p(i nt ), ifro<--l; (7.30) 

and for m > f , with the help of _^ {n ~™^ t 2 < a 2 t t 2 from (7.11), we obtain 
\Hr(w)\ 

(3 3R 3 3R \ 

a: »t J [- 0i nV3 A1 1 (t) + x n 2 t I (^f_^ A1 A (i) + a&I/^ya^ (f) + i„ 2 ( Jr»2/3 vl ^j 

77, 

=: p(x„ t ), if m > - (7-31) 
Thus in either case, by (7.29), we have 

|V>(*)| <exp{ j%! (expi 5 x nt \ p{x nt ) 



7T I \ 7T 



G5 



The second term in the product above is clearly bounded from above by a constant depending on R for all 
x n t, and thus for all < \t\ < ir and all values of a n . The first term exp { — ^er 2 /; 2 } is a decreasing function 
of t. Putting these together, and then using the definition of t from (7.13) yields 

m)\<C R exv\-^a 2 n A=C R (-^=) , t < \t\ < n. (7.32) 



7T^ \ \ Til 



We proceed by bounding |</>(i)|. It is easy to calculate 
\d>(t)\= exp | - J2 Iog(l-^(e«-l) 

t fc=m+l ^ 

We note here that \</>(t)\ is obviously a decreasing function on the whole interval [0, 7r], hence |0(f)| < |0(to) 
for all t < \t\ < tt. 

Now for any k = {m + 1, . . . ,n — 1}, by the right hand side of the inequality 



n , 

/ 1 + 2 Iii^i(l-C0St) 



¥ -^<l-co S t< 0<t<l, 



and (7.14) or (7.16), we get 



77(77 — fc) , , n(n — m — 1) 9 
:= 2 1 - 7 (1 - cos i ) < A TT^tn < 1. 



/v 2 



(m + 1) 



2 "0 



Therefore we can apply the inequality 

1 



1 + x 



<exp<-x+ — > 1 < x < 1, 



with x = Xkt to obtain 



|<K*o)| < ex P {-O'nC 1 - cos *o) + &n(l - COS t ) 2 } , 



where 6„ = n 2 £)£ =m+1 ^ ' Next ' applying ( 7 - 33 ) S ives 



W*>)| < cxp <{ -a 2 > < exp I -a^ [ 1 - 



2 24 



f 2 f 4 ' 2 ~ 



2 30 



1 1 M§ 



2 24 4cr 2 



(7.33) 



(7.34) 



Now we shall prove 6„ig < c 2 . Since the terms in b n = n 2 Y2k=m+i ^ d ecrease as k increases, 
with our usual technique we obtain 



b n < n 
1 



" (n - xf 



1 1 



n 1 



dx = n I — rdx — 2n / —da; + n / — -dx 



1 



1 1 



a; 

1 1 

777 71 



1 



1 fn — 777 

— 77 



By (7.14), (7.16) and the bound above for b T[ 



77 — 777 \ / 777\ 2 1 (?7 — 777)'' 



r, 1 / 77 — 777 \ / 777 \ 



48 77777 



and the latter expression is at most cr 2 by (7.11) and 77 — 777 < n if m < § — 1, and (7.12) and (n — m)/m < 1 
if 777 > 3. Therefore we have 6 n ig < <r 2 , and substituting this into (7.34) yields 



\(f>{t)\ < Cexp 



C 



tn < It < 7T. 



(7.35) 
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Now with the help of (7.35) and (7.32), it is easy to bound the difference \4>(t) — ip{t)\ at the points 
to < \t\ — n i which is necessary for the application of the Theorem 2.2.1: 



m)-m\<m)\ + \m\<c R ^ , t <i*i<7r. 



i - a 



If to > §, then 



Therefore we have 

. R 



R 



(n — to) 



R-l 



n {n—m) n 17 — 2 ' 



771 < - — I" 

W)-m\<< ~"VZL* " 2 ' *o < |*I < tt. (7.36) 



Now we apply Theorem 2.2.1. Condition (2.4) is given by (7.28) and (7.29), thus S = 1, 70 = 0, 
71 = if to < f - 1 and 7l = C fl ^# if m > § , 01 = R+ 1, 7 = 1 and p = Condition (2.5) 

is given by (7.36), so r\ = Cr (^7=) if 771 < ^ — 1 and 7/ = Cr ^z^pr-i if m > We also use Proposition 
7.0.1 to bound er„. It follows that 

which is exactly what we wanted to prove. ■ 

We would like to prove an analogous result of Theorem 7.0.2 in the case when a n ,2 < 1, hence from 
now on we assume that this inequality holds true. In this case |_ a «.,2j = 0, so the characteristic function in 
(7.1) has the form 

^)=exp(- £ Iogfl-^^-l)' 

I k=m+l ^ 

Now we also assume (n — m)/n < 1/(471"), which implies 

77 — k . n — k , ; , . n — m — 1 . ; + . n — m — 1 , , 

— r— M = -p le 1 ' " 1 < T^l c - 1 < -T- * ^ !> 

K fc TO+1 TO+1 

for all < t < 7T. This allows us to expand the logarithmic expression in <f>(t), therefore with w = e 1 ' — 1 
and the notation introduced in (7.2), 



(j)(t) = exp < ^2 a n ,r— > , \t\ < 7T 



We note that a„.i = <r„ — a Kj 2, hence the line above can be rewritten as 

4>(t)= X (t)exp{h(w)}, \t\<ir, (7.38) 
where x{t) is the characteristic function of the Poisson distribution with parameter er„ given in (7.4), and 



h(w) = -a„ i2 w + a n ,2-^- +y^a„, r — . (7.39) 
2 * — ' r 

r=3 

Now we fix an integer R > 3, and following the argument in (7.6), we modify the function cxp{h(w)}: 

exp{h(w)} sa exp{/i H (w)} » fl^w), (7.40) 



67 



where 

2 R 

h R {w) = -a„,2W + a„ j2 — + a n ,r — (7-41) 

and 



/ ~7t,7 

r=3 



3.R-2 , ; , , 

= E (7-42) 

We approximate the distribution /i = T>{W n ^ m + c) with //r, which we define to be the finite signed 
measure on the nonnegative integers whose characteristic function is 

V(t) = X (t)H R (c it -I), teR, (7.43) 

repeating (7.9). But now H R (c ii: — 1) is a polynomial of e 1 ' — 1 of degree 3i? 2 — i?, and the corresponding 
Poisson-Charlier signed measure v R = u R {a^, an/ m , ■ ■ ■ ,Om,m ) is defined by 

(3R 2 -R 
1 + E (-l) r+1 4%a(j,^) ] , j G N, (7.44) 

where a^L is the coefficient of (e 1 * — l) r in H R (e lt — 1), and C r (j, ofj is the r-th Charlier polynomial defined 
in (1.5). 



Theorem 7.0.2 We assume a„_ 2 < 1. -For an arbitrary integer R > 3 i/iere exisi a threshold number n R 
depending on R such that if n > n R , then 

sup - u R {k}\ < C R 



Proof. By (7.19), condition a n2 < 1 implies - — < j- for all n greater than some threshold number n R 
depending on R. We fix integers R > 3 and n > n R . 

As it was the case for Theorem 7.0.1, the key of the proof is Theorem 2.2.1. We apply it with 
the measures \i and v R defined above and <o = n - Recalling the decompositions of the characteristic 
functions corresponding to fi and v R in (7.38) and (7.43), we now give an upper bound for the difference 
| exp{/i(e lt — 1)} — H R {e lt — 1)|, \t\ < tt. For an arbitrary such t, 

| cxp{/ l (e it - 1)} - H R (tP - 1)| < Ax + A 2 , (7.45) 

where the As are the errors resulting from the approximations in (7.40). 
Regarding Ai, we apply the inequality 

|e 21 -e*°| < \ (e^l +e^l) \ Zl - z 2 \, z u z 2 £ C, 

and obtain 

Ai = \exp{h(w)} - exp{h R (w)}\ < -(e^l +e\ hR ^\h{w) - h R (w)\. (7.46) 
From the new definitions of h(w) in (7.39) and that of h R (w) in (7.41), 

i7 f \ i it / m M 2 M r 

\h R [w)\\J \h[w)\ < a n 2\w \+a n ,2—^- + > a„, r 

2 — ' r 

r=3 
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By (7.20), 



\h R (w)\V\h(w)\ < 



(n — to) 3 (n — m) 3 \w\ 2 (n ~ m) \w\ 3 



w 



' ' 2 n 3 

3 qi ( m ^,^4 
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r=3 



W 



r-3 



< 



7r\ (n — ?ti) 32 (n — to) 



w 



w 



7T 32tt 2 \ (ti — to) . 



(7.47) 

At the second inequality we used (n — m )/n < l/(4?r) and \w\ = \c it: - 1| < \t\ < tt. Note that the latter 
inequality and a n ,2 < 1 together with (7.19) imply 



(n — m) 



,2/3 



\w\ < C, 



thus we also have 



\h R (w)\ V \h(w)\ < C. 
If we write (7.48) back into (7.46), we obtain 



(7.48) 



Ai < C\h(w) - h R (w)\ =C 
2 R+1 (n-m) R + 2 . j 



a n ,r — 

r=i?+l 



r=R+l 



< 



R + l n 



R+l 



1 E h 



r-_R-l 



Again, we see from \w\ < ir and (n — m)/n < l/(47r) that the last sum above is finite for all \t\ < tt. We 
conclude 

A! < C R { ^-0^\ W \ R +\ (7.49) 

We now deal with A2. Recalling the definitions in (7.41) and (7.42), by the series expansion of the 
exponential function, we have 



A 2 = I exp{h R (w)} - H 3R _ 2 (w)\ 



< \h R (w)\ 3R -i ™ \h R {w)\ 

iv. ^ 



00 ui 



E 



h R (w) 



l\ 

\ h Ri w )\ 



l=3R-l 
1-3R+1 \h „U,,\\3R-1 



- 1)! (l-3R+l)\ (3R-1)\ 



■exp{\h R (w)\}. 



Now exp{\h R (w)\} < C by (7.48), and 



\h R {wt R - l <C R 



(n — m) 



w 



3R-1 



c R 



(n - m) 3R 
^2R I 



\3R-1 



,2/3 



3(2_R-1) 



where the last fraction is less than some constant depending on R by a„ 2 < 1 and (7.19). Therefore we 
conclude 

™\3ii 

|3fl-l 



A, < C, 



(n - m) 3R 



,2R 



We substitute the inequality above and (7.49) into (7.45), thus with \w\ < \t\, we obtain 



I exp{h(e [t - 1)} - H R (e [t - 1)| < C R 



(n — to) 



R+2 



t 



R+l 



c, 



(n — to) 
TTr 



3R 



3R-1 



Now we apply Theorem 2.2.1. Condition (2.4) with to = tt is given by the last inequality and (7.29), 

which is also true in this case, thus S = 2, 70 = 0, 71 = C R ''""h+i — > 72 = Cr ^"~^r — , &i = R + 1, 
6*2 = 3i? — 1, 7 = 1 and p = -^a 2 . We also use Proposition 7.0.1 to bound a n . It follows that 

1 



sup |^{fc} - u R {k}\ < Cr 



(V^) R - 
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and the proof of Theorem 7.0.2 is complete. ■ 



As outlined at the beginning of the chapter, our goal is to estimate the total variation distance of jj, 
and vr. Formula (2.8) in Theorem 2.2.1 also provides us a way to derive total variation error bounds form 
the local error bounds given by Theorems 7.0.1 and 7.0.2. 

But first, we would like to determine the difference in total variation between the approximating mea- 
sures for successive values of R. By Lemma 6. in [2], we know that the total variation norm of C r (-, ct^)Po(ct^) 

is less than ((2r)/(eer 2 )) . With the help of this and Propositions 7.0.3 and 7.0.1, one can derive that 
the term that has the greatest total variation norm in the sum that defines vr+i — vr by (7.10), is the one 
belonging to r = R + 1, thus 



Wr+i — vr\\tv < CRd n 



R+l 



2 \ fl+i 



a, 



From Proposition 7.0.3, one can also deduce that the order of a^+J is n R+1 /m R , (n — m) 3 /n 2 and (n — 
m) R+2 / n R+1 respectively, in the three cases indicated in the display below. Hence, by Proposition 7.0.1, we 
have 

Cr ^^-^ r _ 1 , in the case of Theorem 7.0.1 when m < t- — 1; 

Wr+i — vr\\tv < { Cr (^2)r- 2 , in the case of Theorem 7.0.1 when m > ^; (7.50) 
Or , in the case of Theorem 7.0.2. 



Corollary 7.0.1 For all n and m for which Theorem 7.0.1 is valid we have 

1 \ .„ n 



and 



<2tv(M: v r) < C R a n log cr„ -= , ifm<- — l, 



2 



R-3 



it 



dTv(v,v R ) < C R - rrr— ifm>-, 

(n — m) H 1 2 

with vr defined in (7.10). For all n and m for which Theorem 7.0.2 is valid we have 

<hv(H,Vn) < Cr , 

with vr defined in (7.44)- 

Proof. First we assume that n and m satisfy the conditions of Theorem 7.0.1. It follows from (2.9) and the 
end of the proof of Theorem 7.0.1 that 



e ab <{ ")^ls ^ 7v™y 2 ' " 2 ' (7.51) 

OR^p^+CR(b-a + 2)^ F=T , ifm>f. 



In order to obtain a total variation bound by (2.8), we need to be able to control the tails of the 
approximating measure vr. As in [6] p. 9, it can be deduced from the Chernoff inequalities for Po(cr^) that 

Wr\{[0, a)} < R 2 i<m ax ^41} exp |- ^'jj } , < a < a\ (7.52) 

and 

M{(6,oo)}<i? 2 max {a£U exp (- (& ~ f ~ ) , a 2 n + R 2 < b < 2a 2 n . (7.53) 
If m < § - 1, then by (7.20), 



max 

Kr<R 2 



70 



and we choose 



and 
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3o-2 log 
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b = al + R 2 
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3o-2 log 



(vM 



R-l 



By (7.11), the argument of the logarithmic expressions in a and b are less than a^ R , hence for all large 
enough a n , < a < er 2 and u\\ + R 2 < b < 2a 2 . In the case when m > §, we have 



max {a^ m } < C R 

l<r<li 2 



(?i — raY 



by (7.20), and we put 



and 



a = tr„ 



1 



3er2 log 



(n — m) 3 \ (n — m) 



ii-2 



.R-3 
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3o-2 log 



(n — ?7i) 3 \ (n — m) 



ni- 



R-2 



R~3 



Again, we see form (7.11) that the argument of the logarithmic expressions in a and b are less than cr^ , 
hence for all large enough <r„, < a < cr 2 and <r„ + i? 2 < 6 < 2cr 2 i . Therefore, in both cases one can apply 
the inequalities (7.52) and (7.53), which yield 



\v R \{[0,a)U(b,oo)}< 



C R 



OR (n-m)"- 1! ' " "" — 2 



if m < f - 1; 
if m > 2. 



Combining this with (7.37) and (7.51) in (2.8), with the choices of a and b given above, one can deduce 



C, 



C R a n log o r , 



^R (n _ m) u-t 



R-l / 1 \ R 

C R a n logg» (i^)^-i , if m > § 



if m < | - 1; 



Together with Proposition 7.0.1 this gives 



d TV (^,v R ) < 



C R a n \oga n (-_) 
C R logej n ^_^ )h _ 2 , 



if m < f - 1; 
if m > 



(7.54) 



We see that in the case when m < 



S — 1, we already have the inequality we aimed for. In the 
latter case, when m > one can omit the log cr n factor in the bound above with the help of the following 
argument. Note that 

d T v(n,v R ) < \ \v R +i - v R \\tv + d-Yv{pL,v R+ i). 

The first term on the right hand side of the inequality above can be bounded by (7.50) and the second 
can be estimated by (7.54) with R replaced by R + 1. Then, by Proposition 7.0.1, the second inequality of 
Corollary 7.0.1 follows. 

Now we assume that n and m satisfy the conditions of Theorem 7.0.2. In this case we can apply 
Theorem 3.2. from [6] to obtain 

dTv(n,v R ) < C R \oga n R+1 . 
We can apply the same argument as above to omit the logcr n from the bound above. Indeed, we obtain 



-tTV 



(M, vr) < C R 
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C R log cr„ 



R+2 ' 



Since logcr„ < a n < \/2(n — m)/y/n by Proposition 7.0.1, and in — m)/n < 1, we finished the proof of the 
third inequality of Corollary 7.0.1. ■ 

Comparing the results of Corollary 7.0.1 with (7.50), we see that in the small m case, when m < n/2—1, 
our results are not optimal in the sense that the error order of the approximation with vr does not coincide 
with the order of the total variation norm ll^/j+i — ^_r||tv of the (R + l)-th correction term. We see 
from the proof of Corollary 7.0.1, that the technique used in the other cases to omit the logcr„ from the 
total variation bounds does not work for small m, because for such m, a n is of order n/\frn, which is 
not comparable with the l/(i/m) fl+1 factor of the error order. It is an interesting open problem whether 
cItv(h, V R) < Cr/ {\/rn) R ~ 1 can be achieved for m < n/2 — 1. 

We finish the chapter by comparing its results with the ones obtained in Chapter 6 for compound 
Poisson approximation. First, we note that with some extra, but trivial considerations, the proofs of the 
chapter could be modified to hold true for R = 2 also. Now, if the hypothesis formulated in the previous 
paragraph is true, than the bounds we obtain for d-rv(n, v-i) would exactly match the total variation bounds 
in (6.8) for the compound Poisson approximation of /i. We guess that if one defines \ in (7.4) to be the 
characteristic function of the compound Poisson distribution given in Theorem 6.2.1, then the technique used 
in this chapter with this more sensitive choice of approximating measure would lead to an improvement in 
terms of the error bounds. 
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Summary 



The coupon collector's problem is one of the classical problems of probability theory. In this thesis, we are 
interested in the version of the problem, when a collector samples with replacement a set of n > 2 distinct 
coupons so that at each time any one of the n coupons is drawn with the same probability 1/n. For a fixed 
integer m G {0, 1, . . . , n — 1}, this is repeated until n — m distinct coupons are collected for the first time. 
Let W ntm denote the number of necessary repetitions to achieve this. Thus the random variable W n ^ m , 
called the coupon collector's waiting time, can take on the values n — m, n — m + 1, n — m + 2, . . ., and gives 
the number of draws necessary to have a collection, for the first time, with only m coupons missing. In 
particular, W n fi is the waiting time to acquire, for the first time, a complete collection. 

Different limit theorems have been proved for the asymptotic distribution of W njm , depending on how 
m = m(n) behaves as n — > oo. (All asymptotic relations throughout are meant as n — > oo.) The first result 
was proved by Erdos and Renyi for complete collections when m = for all n G N, obtaining a limiting 
shifted Gumbel extreme value distribution. This result was extended by Baum and Billingsley, who examined 
all relevant sequences of m = m(n). They determined four different limiting distributions: the degenerate 
distribution at 0, the Poisson distribution, the normal distribution and a Gumbel-like distribution. 

One of the aims of this thesis is to refine the limit theorems of Baum and Billingsley. Our basic goal 
is to approximate the distribution of the coupon collector's appropriately centered and normalized waiting 
time with well-known measures with high accuracy, and in many cases prove asymptotic expansions for the 
related probability distribution functions and mass functions. The approximating measures are chosen from 
five different measure families. Three of them - the Poisson distributions, the normal distributions and the 
Gumbel-like distributions - are probability measure families whose members occur as limiting laws in the 
limit theorems of Baum and Billingsley. 

The fourth set of measures considered is a certain : fi > 0, a > 0} family of compound Poisson 

measures. For each [i > and a > 0, we define 7r M a to be the probability distribution of Z\ + 2Z2, where 
Z\ and Zi are independent random variables defined on a common probability space, Z\ ~ Po(/i) and 



The fifth set of approximating measures we consider is the family of Poisson-Charlier signed mea- 



Z 2 ~ Po(a/2). 
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Our results are the following: 



Chapter 3 

In this chapter, we are interested in the the asymptotic behavior of the distribution function of the 
appropriately standardized waiting time F n ^ m , if to is a fixed constant for all n and n — > oo. With 
F m denoting the limiting distribution function, for every to, we give a one-term asymptotic expansion 
F m + G n ^ m that approximates F n ^ m with the uniform order of 1/n such that the explicit sequence of 
functions G n>m has the uniform order of (log n)/n. We use characteristic functions in our proof. 

We also give an argument, that not only proves that the error order of this approximation is sharp, 
but also that no longer asymptotic expansion of F n>m can improve the error order of 1/n. 

Chapter 4 

In this chapter, with a classical characteristic function method, we prove that the error order for normal 
approximation to the coupon collector's standardized waiting time is at most n/(ma n ) in Kolmogorov 
distance, where er„ denotes the standard deviation of the waiting time. One can check that this bound 
is ideal in the sense that it tends to 0, iff n and to satisfy the conditions of the central limit theorem 
concerning the coupon collector's problem. 

Chapter 5 

In the first section of Chapter 5, we consider Poisson approximation to the distribution of sums 
of asymptotically negligible integer valued random variables in general. We complement a classical 
Poisson convergence theorem of Gnedenko and Kolmogorov. Considering an arbitrary triangular array 
{l^i, Y n 2 7 . . . , Y nr . n } n £p$ of row- wise independent nonnegative integer valued random variables, for 
each n, we approximate the distribution of the n-th row sum with a Poisson distribution whose 
mean A„ is defined only in terms of the distributions of the random variables in the n-th row, but we 
do not assume the existence of moments. We give both lower and upper bounds, which have precisely 
the same form, up to a constant, provided that the means A„ are bounded away from infinity. We thus 
refine the obvious approximation of the Y n -s that the Gnedenko-Kolmogorov limit theorem suggests. 

In the next section we examine how the coupon collector's problem fits in the framework of the previous 
section. We show that the Poisson limit theorem concerning the coupon collector's waiting time is a 
special case of the Gnedeno-Kolmogorov theorem. Applying the general results of the previous section 
to the waiting time, we obtain a Poisson approximation of error order 1/i/n. 

In the third section of the chapter we take advantage of the combinatorial structure of the coupon 
collector's problem. This combinatorial approach yields us a stronger result than the one of the 
previous section: we derive the first asymptotic correction of the P(W„ TO — (n — to) = k), k = 0, 1, . . ., 
probabilities to the corresponding Poisson point probabilities. 

In the final section of Chapter 5, we approximate the coupon collector's shifted waiting time W n ,m = 
W n ,m — (n — to) with another Poisson law, namely with the one that has the same mean as W n>m - 
One can easily calculate that in the range of parameters n and to for which the Poisson limit theorem 
holds true, the error order of this new approximation is 1/n, which is clearly better than the error 
order \j \fn given in the preceding two sections for the same case. The proof here is based on Stein's 
method, and heavily uses the fact that the means of the compared probability measures coincide. 

Chapter 6 

In the first section of Chapter 6 we consider translated compound Poisson approximation of sums of 
independent integer valued random variables in general. Using Stein's method, Barbour and others 
gave bounds for the errors of such approximations in total variation distance. Their upper bounds are 
expressed with the help of the first three moments of the summands X\ , Xi , . . . , X n and the critical 
ingredient d TV {V (W„) , V {W n + 1)), where W n = £"=i X r 

The expression g?tv (T> (W n ) , T> (W n + 1)) is usually bounded by the Mineka coupling, which typically 
yields a bound of order 1 / ^Jn. If the Xj 's are roughly similar in magnitude, this is comparable with the 
order 0(1 / VVarW n ) expected for the error in the central limit theorem. However, if the distributions 
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of the Xj become progressively more spread out as j increases, then VarW n may grow faster than n, 
and then l/\fn is bigger than the ideal order 0(1/ \/VarW n ). In fact, this is the situation in the case 
when we chose W n to be the coupon collector's waiting time. We introduce a new coupling which 
allows us to improve the Mineka bounds in such settings. 

In the next section, we approximate the distribution of the appropriately centered coupon collector's 
waiting time with a compound Poisson measure 7r M . a defined above. We apply general results of 
translated compound Poisson approximation of sums of independent integer valued random variables 
and our new coupling. We prove that a translated compound Poisson approximation to the collector's 
waiting time, with ideal error rate, can be obtained in all ranges of n and m in which a central 
or Poisson limit theorem holds true. Comparing this result with the ones we obtained for normal 
approximation, we see that the same or better order of approximation is obtained with this discrete 
approximation, and now with the error measured with respect to the much stronger total variation 
distance. 

• Chapter 7 

In the final chapter of the thesis we approximate the coupon collector's shifted waiting time W njn = 
W njm — (n — to) with Poisson-Charlier signed measures in total variation distance. To do so, we 
apply a characteristic function technique. For an arbitrary R > 3, we choose a Poisson-Charlier 
signed measure i/r depending on R. Approximating W n , m with Vr, we obtain error bounds of order 
a n logcr n (l/v / m) fl , (y / n)- R ~ 3 /(n — m) R ~ 2 and (n — m) / (^/n) R+1 depending on how the sequences m/n 
and er^ — /i_ (n — to) behave as n tends to infinity, where /i„ is the mean and er^ is the variance of 
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Osszefoglalas 



A kupongyujto problema a valoszinusegszamitas egyik klasszikus problemaja. A dolgozatban a problema 
kovetkezo valtozataval foglalkozunk: adva van n > 2 kiilonbozo kupon, melyekbol egy gyujto veletlenszeru 
visszateveses mintat vesz ugy, hogy minden egyes alkalommal az n kupon barmelyiket azonos, tehat 1/n 
valoszinuseggel szerzi meg. Valamely rogzitett m G {0, 1, ...,n — 1} eseten a mintavetelt addig folytatja, 
amlg eloszorre pontosan n — m kiilonbozo kupont nem gyujtott. Jelolje W nm az ehhez sziikseges ismetlesek 
szamat. Tehat a W n ^ m veletlen valtozo. amelyet a kupongyujto varakozasi idejenek nevcziink, az n — 
m, n — m + 1, n — m + 2, . . . ertekeket veheti fel, es megadja, hogy a gyujtonek hanyszor kell huznia ahhoz, 
hogy m darab kupon kivetelevel minden kupon a birtokaban legyen. Specialisan, W n fi a teljes gyujtemeny 
megszerzesehez sziikseges varakozasi ido. 

Kiilonbozo hatareloszlas teteleket bizonyitottak W n . m aszimptotikus eloszlasara attol fiiggoen, hogy 
az m = m(n) sorozat hogyan viselkedik, amint n — > oo. (A tovabbiakban minden konvergencia es aszimp- 
totikus relacio n — > oo mellett ertendo.) Az elso eredmeny Erdos es Renyi nevehez fuzodik, akik a teljes 
gyujtemeny esetere, amikor minden n € N eseten m = 0, eltolt Gumbel-eloszlast kaptak hatareloszlaskent. 
Ezt az eredmenyt altalanositotta Baum es Billingsley, akik minden m = m(n) sorozat tipust vizsgaltak. 
Negy kiilonbozo hatareloszlast hataroztak meg: a 0-ra koncentralt eloszlast, a Poisson eloszlast, a normalis 
eloszlast, es egy a Gumbel-eloszlasbol szarmaztathato eloszlast. 

A dolgozatban finonn'tjuk a fenti, Baum es Billingsley nevehez fuzodo hatareloszlas teteleket. Celunk 
a megfeleloen centralizalt es normalizalt varakozasi id5 eloszlasanak jol ismert mertekekkel torteno minel 
pontosabb kozelitese, es sok esetben a kapcsolodo eloszlasfiiggvenyek es valoszinusegi pontfiiggvenyek asz- 
imptotikus sorfejtese. A kozelito mertekeket 6t kiilonbozo mertekcsaladbol valasztjuk. Ezek koziil harom - a 
Poisson eloszlasok, a normalis eloszlasok es a Gumbel-tipusii eloszlasok - olyan mertekcsaladok, melyeknek 
tagjai hatareloszlasakent szerepelnek Baum es Billingsley teteleiben. 

A negyedik approximalo mertekcsalad az osszetett Poisson eloszlasoknak bizonyos {n^ lM : fi > 0, a > 
0} osztalya. Tetszoleges fi > es a > eseten 7r^. a a Z\ + IZ2 veletlen valtozo eloszlasat jeloli, ahol Z\ es 
Z 2 valamely kozos valoszinusegi mezon definialt fiiggetlen veletlen valtozok, Z\ ~ Po(/i) es Z 2 ~ Po(a/2). 
Az otodik kozelito mertekcsalad a Poisson-Charlier elojeles mertekek osztalya. Tetszoleges pozitiv valos 
A, a^ 1 ', . . . ,0,^ es S e N eseten a v = ^(AjS^ 1 ^, . . . ,a^) Poisson-Charlier elojeles mertek az az elojeles 
mertek, amely a nemnegativ egeszekre van koncentralva, es = Po(A) ^X)r=i( — l) r Q' ( ''*' ) Cr(j, A) J , j € N, 
ahol C r (j,X) az r-edik Charlier polinom. 
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Eredmenyeink a kovetkezoek: 

3. fejezet 

A 3. fejezetben a megfeleloen standardizalt varakozasi ido F n m eloszlasfiiggvenyenek aszimptotikus 
viselkedeset vizsgaljuk abban az esetben, amikor m minden n-re rogzitett konstans esn-> oo. Jelolje 
F m a hatareloszlas fiiggvenyt. Minden m eseten olyan F m + G n<m egytagu aszimptotikus sorfejtest 
adunk, amely egyenletesen 1/n rendben kozeliti az F n ^ m eloszlasfiiggvenyt, tovabba az explicit modon 
megadott G n>m fiiggvenyek sorozata egyenletesen (log n)/n rendu. A bizonyitasban karakterisztikus 
fiiggvenyeket hasznalunk. 

Azt is belatjuk, hogy a kozelites hibarendja eles, es hogy i^^-nek semmilyen hosszabb aszimptotikus 
sorfejtese eseten nem kaphatunk az 1/n rendnel kisebb hibarendet. 

4. fejezet 

A dolgozat ezen fejezeteben klasszikus, karakterisztikus fiiggvenyeket hasznalo modszerrel belatjuk, 
hogy a kupongyujto varakozasi idejenek normalis eloszlassal valo kozeh'tesenek Kolmogorov tavolsagban 
inert hibaja legfeljebb n/(mcr n ) rendu, ahol er„ a varakozasi ido szorasat jeloli. Ellenorizheto, hogy 
ez az approximacio jo abban az ertelemben, hogy 0-hoz tarta, ha n es m teljesitik a kupongyujto 
problemara vonatkozo centralis hatareloszlas tetel felteteleit. 

5. fejezet 

Az 5. fejezet elso reszeben altalanos fiiggetlen nemnegativ egeszerteku veletlen valtozok osszegeinek 
eloszlasat kozelitjiik Poisson eloszlassal. A Gnedenko es Kolmogorov nevehez fuzodo klasszikus Poisson 
hatareloszlas tetelt finomitjuk. Tetszoleges, soron-kent fiiggetlen, nemnegativ egeszerteku veletlen 
valtozokbol alio {Y n i, Y n 2, . . . , Y nrn } ne ^ szeriasorozatot tekintve minden n eseten az n-edik sorosszeg 
eloszlasat olyan Poisson eloszlassal kozelitjiik, melynek A„ varhato erteke csak az adott sorban szereplo 
valtozok eloszlasatol fiigg, de nem koveteljiik meg momentumok letezeset. A kozelites hibajara also 
es felso korlatot is adunk, melyek rendje konstans szorzotol eltekintve megegyezik, felteve, hogy a X n 
parameterek korlatosak. Ezaltal jobb kozeh'teset adjuk az Y n valtozoknak, mint amit a kezenfekvo, 
hatareloszlassal torteno approximacio jelent. 

A fejezet masodik alfejezeteben megmutatjuk, hogy a varakozasi idore vonatkozo Poisson hatarelos- 
zlas tetel specialis esete a Gnedeno-Kolmogorov tetelnek. Megmutatjuk, hogy ha az elozo alfejezet 
eredmenyeit alkalmazzuk a varakozasi idore, l/y/n hibarendu Poisson kozeliteset kapjuk. 

A harmadik alfejezetben a kupongyujto problema kombinatorikai struktiirajara epitiink. A kombi- 
natorikai megfontolasokra tamaszkodo modszer segitsegevel erosebb eredmenyt tudunk igazolni, mint 
az elozo alfejezetben: a P(W n , m — (n — m) = k), k = 0,1,..., valoszmusegek megfelelo Poisson 
valoszinusegekkel torteno kozeliteset pontositjuk az elso korrekcios tag meghatarozasa reven. 

Az 5. fejezet utolso alfejezeteben a kupongyujto W n . m = W n ,m — (n — m) eltolt varakozasi idejet egy 
ujabb Poisson^ eloszlasu veletlen valtozoval kozelitjiik, meghozza olyannal, amelynek varhato erteke 
megegyezik W n ^ m varhato ertekevel. Kiszamolhato, hogy azon n es m parameter ertekek eseten, 
melyekre ervenyes a Poisson hatareloszlas tetel, az approximacio hibajanak rendje ami vilagos, 
hogy kisebb, mint az elozo ket alfejezetben ugyanezen esetre bizonyitott l/-y/n-es hibarend. A bi- 
zonyitas a Stein-modszeren alapszik, es kihasznalja azt a tenyt, hogy az osszehasonlitott eloszlasok 
varhato ertekei egyenloek. 

6. fejezet 

A hatodik fejezet elso alfejezeteben fiiggetlen egeszerteku veletlen valtozok osszegeinek osszetett Pois- 
son eloszlasu valtozokkal torteno kozeliteset vizsgaljuk altalanosan. A Stein-modszer segitsegevel 
Barbour es masok felso korlatokat adtak az ilyen tipusu approximaciok teljes variacios tavolsag- 
ban mert hibaira. Ezek a korlatok az Xi, X2, X n osszeadandok elso harom momentumanak es a 
d T v {V (W n ) , V (W n + 1)) kifejezesnek fiiggvenyei, ahol W n = YT 3 =\ x r 

A cIty (T> (W n ) ,T> (W n + 1)) kifejezest altalaban a Mineka-csatolas segitsegevel lehet becsiilni, ami 
tipikusan l/\/n rendu eredmenyt ad. Ha az Xj veletlen valtozok nagyjabol azonos szorasuak, akkor 
ez az eredmeny kozel van a centralis hatareloszlas tetel eseten elvart 0(1 /-^/Var W n ) hibarendhez. 
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Azonban ha az Xj veletlen valtozok eloszlasai egyre laposabbak, amint j no, akkor ~VarW n nohet gy- 
orsabban, mint n, es ekkor l/\fri joval nagyobb lesz, mint az elvarhato 1 / \/Var W n -es rend. Pontosan 
ez a helyzet, ha W„-nek a kupongyujto varakozasi idejet valasztjuk. Bevezetiink egy uj csatolast, 
mely segitsegevel ilyen esetekben jobb eredmenyeket tudunk bizonyftani, mint a Mineka-csatolas segit- 
segevel. 

A kovetkezo alfejezetben a kupongyujto megfeleloen centralizalt varakozasi idejenek eloszlasat a ko- 
rabban definialt ir^^ osszetett Poisson eloszlassal kozelitjiik. Fiiggetlen egeszerteku veletlen valtozok 
osszegeire vonatkozo altalanos osszetett Poisson approximacios eredmenyeket es az uj csatolasunkat 
alkalmazzuk. Belatjuk, hogy a W n ,m varakozasi ido jol kozelitheto osszetett Poisson eloszlassal abban 
az esetben, amikor az n es m parameterek teljesitik a kupongyujto problemara vonatkozo centralis 
vagy Poisson hatareloszlas tetel felteteleit. Ezeket es a normalis approximaciora kapott eredmenyeket 
osszehasonlitva latjuk, hogy az itt bevezetett diszkret approximacio ugyanolyan, vagy jobb kozeliteset 
jelenti a varakozasi idonek, mint a normalis approximacio. Raadasul a kozelites hibajat itt a Kol- 
mogorov tavolsagnal sokkal erosebb teljes variacios tavolsagban merjiik. 

• 7. fejezet 

Az utolso fejezetben a kupongyujto W n ,m = W n>m — [n — m) eltolt varakozasi idejet Poisson-Charlier 
elojeles mertekekkel kozelitjiik teljes variacios tavolsagban. Ehhez egy karakterisztikus fiiggvenyeket 
hasznalo modszert alkalmazzuk. Tetszoleges R > 3 eseten definialunk egy i?-tol fiiggo vr Poisson- 
Charlier elojeles merteket. Az eltolt varakozasi idot ezzel kozelftve o n log cr n (l / 1 y/m) R , (- v /n) fl_3 /(n — 
m) R ~ 2 , illetve (n— m) / (v^)^^ 1 hibarendeket kapunk aszerint, hogy azm/n. illetve a o\ — [i n — (n—m) 
sorozatok hogyan viselkednek, amint n ~ > oo, ahol [i n a varakozasi ido varhato erteket, <j\ pedig a 
szorasnegyzetet jeloli. 
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